Every user's app data lives in ~/Library, hidden by default. Inside it:

On WindowsOn a Mac
%APPDATA% (Roaming)~/Library/Application Support
Registry settings for an app~/Library/Preferences/com.maker.app.plist
%LOCALAPPDATA%\Temp, caches~/Library/Caches
Event logs~/Library/Logs, and the Console app
%PROGRAMDATA%/Library (for all users)

To open it once: hold , open Finder's Go menu, and Library appears in the list. To keep it visible: open your home folder, press ⌘J for the view options, and tick Show Library Folder.

Deleting an app leaves its Application Support, Preferences and Caches behind, usually a few megabytes. Safe to delete after the app is gone, except for anything you want to keep, such as a mail client's local mailboxes.