Music Recommender

A Bay Area-first music discovery engine that starts from upcoming local shows, then helps you figure out what to listen to next.

Current scope

• Ingest local shows from eleven Bay Area sources via the structured crawler on localhost:11235: 19hz (dated + recurring tables), Foopee, SanFran666co, DoTheBay, FunCheap, The Knockout, Feight (F8), The Midway, SFStation, SF Rec & Park, and Resident Advisor
• Capture editorial recommendation signals from NPR, The Needle Drop, and KEXP
• Store artists, venues, events, genres, images, and source evidence in SQLite
• Enrich artists with Every Noise and MusicBrainz genre data
• Produce genre-first artist recommendations using simple likes and dislikes
• Import persistent loved-artist assertions from CSV and use them as cautious taste signals
• Record attended shows directly from the CLI or CSV with artist/venue resolution and unresolved review support
• Use stored loved-artist and attended-show history as light, explicit taste signals in recommendation scoring

Quick start

cargo run -- init-db
cargo run -- doctor
cargo run -- musicbrainz-health
cargo run -- add-loved-artist Broadcast --source manual --note "favorite band"
cargo run -- import-loved-artists ./loved-artists.csv --default-source manual
cargo run -- list-loved-artists
cargo run -- add-attended-show --artist Broadcast --venue "Fox Theater" --source manual
cargo run -- import-attended-shows ./attended-shows.csv --default-source history.csv
cargo run -- list-attended-shows --unresolved-only
cargo run -- completions --all-shells --output-dir ./completions
cargo run -- ingest-all --artist-limit 500
cargo run -- recommend --liked-genre punk --disliked-genre edm

Notes

• The app expects the crawler service at http://localhost:11235/crawl
• The app now defaults to XDG locations: ~/.cache/music-recommender, ~/.config/music-recommender, and ~/.local/share/music-recommender
• On first run, the app writes a config template to ~/.config/music-recommender/config.toml
• Set share_secret in that config file to a long random value; share URLs use it for token signing
• Point the app at MusicBrainz by setting ordered full endpoints in musicbrainz_base_urls, including /ws/2 in each URL
• crawl4ai_urls in config is an ordered fallback array
• Set searx_search_urls and crawl4ai_urls in ~/.config/music-recommender/config.toml; arrays are tried in order
• The default SQLite database lives at ~/.local/share/music-recommender/music-recommender.sqlite3
• On first run, the bundled Every Noise seed file is copied into ~/.local/share/music-recommender/everynoise_genres_20260317_135315.json unless everynoise_path is set in config
• Crawler and MusicBrainz responses are cached under ~/.cache/music-recommender
• Show crawler responses default to a 24 hour TTL and signal crawler responses default to a 48 hour TTL
• Configure crawler TTLs with crawler_show_cache_ttl_secs and crawler_signal_cache_ttl_secs
• The crawler also has configurable timeout and retry settings: crawler_timeout_secs, crawler_max_retries, and crawler_retry_base_delay_ms
• Set musicbrainz_user_agent in ~/.config/music-recommender/config.toml; use a real contactable value like music-recommender/0.1.0 (you@example.com)
• MusicBrainz lookups retry transient failures; defaults use 1100ms request spacing and parallelism 1 for every endpoint type
• cli_executable remains a scheduler setting because music-server launches root CLI stages for scheduled work
• Tune MusicBrainz behavior with musicbrainz_min_interval_ms, musicbrainz_max_retries, musicbrainz_retry_base_delay_ms, and musicbrainz_parallelism
• Use cargo run -- doctor to print resolved config and endpoint reachability
• Use cargo run -- musicbrainz-health to run a fast MusicBrainz-specific probe that validates both artist search and artist lookup with usable genre data
• Use cargo run -- import-loved-artists ./loved-artists.csv --default-source manual to store explicit loved-artist assertions from CSV and immediately try to enrich them with MusicBrainz genres
• Use cargo run -- add-loved-artist Broadcast --source manual for quick one-off loved-artist entry without preparing a CSV first
• Use cargo run -- list-loved-artists to inspect the stored per-source loved-artist assertions, including notes, confidence, and any persisted MBID
• Use cargo run -- add-attended-show --artist "Visages" --venue "Public Works" --source manual to log attended shows directly from the CLI
• Use cargo run -- import-attended-shows ./attended-shows.csv --default-source history.csv to import attended-show history from CSV using an artists column split by |
• Use cargo run -- ingest-signals --source kexp --refresh to ingest a broader recent KEXP review set plus a recent KEXP playlist snapshot
• add-attended-show tries exact local resolution first, then MusicBrainz for artists; unresolved venue/artist names are preserved instead of discarded
• import-attended-shows uses the same resolution path and also merges likely duplicates so a corrected typo re-import can land on the earlier unresolved row instead of creating a second copy
• Use cargo run -- list-attended-shows --unresolved-only to review typo-prone or still-unmatched attended-show rows and see likely local candidates
• KEXP review mentions and KEXP playlist plays are now stored as separate editorial sources, so recommendation buzz/source diversity can treat them comparably to NPR and Needledrop support
• KEXP review and playlist support can also weakly reinforce an artist's already-strong genres during enrichment, so KEXP affects genre overlap as well as editorial buzz
• If the local crawler is down, ingest-signals --source kexp now warns and continues with the playlist snapshot instead of failing the whole KEXP ingest
• Use -v or --verbose on any command to print detailed request/progress output
• imported loved artists and resolved attended-show artists now feed recommend automatically as light exact-match and cautious genre-overlap signals
• recommend now defaults to compact output; add --explain if you want the full accumulated reason text
• recommend now caches results under ~/.cache/music-recommender/recommend for 15 minutes and invalidates when recommendation-relevant data changes
• Use cargo run -- recommend --refresh to force a recompute, or cargo run -- recommend --no-cache to bypass the recommendation cache entirely
• Use cargo run -- recommend --after 10d --before 30d to limit recommendations to shows happening within a relative day window
• Use cargo run -- recommend --after 2026-05-01 --before 2026-06-30 to filter by absolute show-day bounds
• --after and --before work at the local event-day level and combine with --horizon-days; invalid ranges where --after resolves later than --before fail fast
• Use cargo run -- completions --all-shells --output-dir ./completions to generate Bash, Zsh, Fish, Elvish, Nushell, and PowerShell completions plus an install guide
• Use cargo run -- completions --shell zsh --stdout to print a single shell script directly to stdout
• Use cargo run -- ingest-all --artist-limit 500 to run ingest-shows, ingest-signals, and enrich-artists in one command
• Add --check-musicbrainz to enrich-artists or ingest-all to run the fast MusicBrainz preflight before any MusicBrainz-backed work starts and fail early if it does not pass
• ingest-all fails fast by default; add --allow-signal-ingest-failures if you want it to continue into enrichment when editorial signal fetching fails
• Use --cache-ttl-hours <n> on ingest-shows or ingest-signals to override crawler TTLs for that run
• Use --show-cache-ttl-hours <n> and --signal-cache-ttl-hours <n> on ingest-all to override stage-specific crawler TTLs
• Use --cache-forever on ingest-shows or ingest-signals to reuse the last crawler response regardless of age
• Use --show-cache-forever and --signal-cache-forever on ingest-all to reuse stale cache for those stages
• Use --refresh to skip cache reads and write fresh responses, or --no-cache to skip both reads and writes entirely
• The Rust implementation avoids unsafe

Docker deployment

Direct ./mk scripts are deployment source of truth. Put share_secret, endpoint arrays, and portable paths in ~/.config/music-recommender/config.toml; direct Docker persists app data, cache, and config under XDG directories. Local and container processes see those directories at identical absolute paths. Crawl4AI is external and is never managed by this project. Daemon settings also live in TOML. mk/run uses CLI arguments only for direct container bind, host port publishing, and server-to-daemon routing.

./mk/run 8000 local 0
curl -fsS http://localhost:8000/healthz
docker logs --follow --tail=200 music-recommender-local-8000

Deployment command:

• ./mk/run [port] [mode] [rebuild] — build and start music-server and download-daemon

Stop or inspect those two containers directly with Docker. Do not stop or manage external Crawl4AI from this project.

One-shot operations use explicit maintenance commands and direct Docker:

./mk/maintenance cli init-db
./mk/maintenance cli ingest-all --artist-limit 500
./mk/maintenance server rebuild-genre-layout
./mk/maintenance api GET /api/daemon-dashboard

First Cutover (2026-08-12)

Before starting new direct Docker services, edit ~/.config/music-recommender/config.toml. Use host-portable paths. Application expands leading ~ in TOML; shell expansion is not involved:

share_secret = "replace-with-a-long-random-value"
database_path = "~/.local/share/music-recommender/music-recommender.sqlite3"
downloads_dir = "~/.local/share/music-recommender/download-daemon/downloads"
everynoise_path = "~/.local/share/music-recommender/everynoise_genres_20260317_135315.json"
searx_search_urls = ["http://your-searx-host:8080/search"]
crawl4ai_urls = ["http://localhost:11235"]

Direct Docker binds each host XDG application directory to the same absolute path in containers. Do not use /data, /cache, or /config as deployment paths.

Stop old app and daemon containers before conversion. Conversion creates timestamped SQLite/config backups, repairs historical schemas, normalizes old searx_search_url to searx_search_urls, and writes a completion marker only after verification:

docker stop music-recommender-local-8000 music-recommender-download-daemon-local-8000
./mk/maintenance cli convert-data --confirm
./mk/run 8000 local 0
curl -fsS http://localhost:8000/healthz

Do not run conversion while app or daemon writes data. If conversion fails, no completion marker is written; use its rollback paths before retrying.

Use distinct port values for separate direct deployments. Configure Soulseek credentials and wait port in TOML, or set SOULSEEK_WAIT_PORT for direct host port publishing. For cross-host moves, stop services, copy the XDG config/data/cache directories, preserve their paths on destination host, then run the same cutover commands. See docs/2026-08-12-config-first-cutover.md.

Listing artists without downloads or artwork

Artists without downloads

sqlite3 ~/.local/share/music-recommender/music-recommender.sqlite3 "
SELECT a.name FROM artists a
WHERE NOT EXISTS (
  SELECT 1 FROM acquisition_artifacts aa
  WHERE aa.artist = a.name
    AND aa.status IN ('downloaded','organized','promoted')
)
ORDER BY a.name;
"

Artists without artwork

sqlite3 ~/.local/share/music-recommender/music-recommender.sqlite3 "
SELECT a.name FROM artists a
WHERE NOT EXISTS (
  SELECT 1 FROM artist_images ai
  WHERE ai.artist_id = a.id AND ai.kind = 'artist_portrait'
)
ORDER BY a.name;
"

Or use the CLI:

music-recommender images missing --kind artist --limit 200

Manual inserts

Artwork — attach from a local file or URL:

music-recommender images attach --kind artist --target "Artist Name" --source manual /path/to/image.jpg

Downloads — submit a job to the download daemon:

music-recommender daemon submit ytdlp '{"url":"https://soundcloud.com/..."}' --kind ad-hoc --priority 20

Both are also handled automatically: artwork resolves on rebuild via images backfill, and planned genre samples download via genre-samples acquire-planned.