public

Scrobbling DJ sets into Last.fm with less than 30 lines of Python

I've recently gotten into following the tracks I'm listening to using Last.fm. One giant gap I've discovered in my everyday listening experience is that it's not easy to import all of the songs played in a DJ set into your Last.fm account, so I've decided to build one.

Latest Post Screenshots as a layer of interoperability by Aleksandar Todorović public

I've recently gotten into following the tracks I'm listening to using Last.fm . While it provides an impressive amount of featured and non-featured ways of tracking the music you've listened to, one giant gap I've discovered in my everyday listening experience is that it's not easy to import all of the songs played in a DJ set into your Last.fm account, so I've decided to build one.

How it works

There's this awesome website called 1001tracklists , in which a community tries its best to track down every single track played in a DJ set. It has become a de-facto standard way of sharing a tracklist, and plenty of DJs just link to it instead of bothering to write down all of the tracks they've played in a set.

This approach is pretty versatile, since it doesn't matter if the DJ uploaded its set on YouTube, Mixcloud, Soundcloud, or livestreamed it via Twitch. In all of those cases, odds are big that you're going to find a tracklist for it on 1001tracklists.

So, the process of getting a set into Last.fm looks like this:

  1. Listen to a recording of a set by your favourite DJ.
  2. Find the list of tracks played in a set using 1001tracklists.
  3. Run the script with the 1001tracklists URL.
  4. Scrape artist names and track names from the site.
  5. Use Last.fm's API to mark the tracks as played (scrobbled) in your Last.fm account.

Of course, this isn't achievable in less than 30 lines of Python without using some libraries that will make our code easier to write, so let's see what could help us out.

Useful Python libraries

The latter three are pretty commonly used when you want to scrape websites with Python. While there are alternatives, getting yourself familiar with these will provide you with an ability to scrape pretty much anything you want, as well as understand somebody else's scrapers. If you'd like to learn them, I'd recommend Chapter 12 from the book Automate the Boring Stuff with Python . You can read it for free on the book's website .

You can install all of these prerequisites with pip install pylast requests fake-useragent beautifulsoup4.

Actual script

The code looks pretty clean compared to other scrapers I've written and used. I'm also posting it on my GitLab as a snippet .

#!/usr/bin/python3

import pylast, requests, sys, time
from fake_useragent import UserAgent
from bs4 import BeautifulSoup

LASTFM_API_KEY = "<your Last.fm API key goes here>"
LASTFM_API_SECRET = "<your Last.fm API secret goes here>"
LASTFM_USERNAME = "<your Last.fm username goes here>"
LASTFM_PASSWORD = pylast.md5("<your Last.fm password goes here>")

url = str(sys.argv[1])
print("Scrobbling from " + url )
network = pylast.LastFMNetwork(api_key=LASTFM_API_KEY, api_secret=LASTFM_API_SECRET, username=LASTFM_USERNAME, password_hash=LASTFM_PASSWORD)

r = requests.get(url, headers={'User-Agent': UserAgent().firefox })
soup = BeautifulSoup(r.text, features="lxml")

for track in soup.find_all('span', class_="trackFormat"):
  full_text = track.text.split(" - ")
  artist = full_text[0][1:]
  track_name = full_text[1][:-1]
  if artist != "ID" and track_name != "ID" and "ID Remix" not in track_name:
    last_api_call = network.scrobble(artist, track_name, int(time.time()))
    print ("Scrobbled: " + artist + " - " + track_name)

You'd need to generate a new API key on Last.fm from here , fill in the four variables in the script with your account info and your API credentials. After doing so, you just run the script like this:

python scrape.py https://www.1001tracklists.com/tracklist/td181gt/rl-grime-jawns-rossy-sable-valley-stream001-united-states-2020-05-30.html

Within these lines, we got rid of some quirks while scraping the output from 1001tracklists:

After running it, the history in your Last.fm account will look something like this:

Note: I ran the script before I've added an exception for "ID Remix". 

The actual HTML code of an identified track on 1001tracklists looks something like this once you prettify it:

<span class="trackFormat"> 
	<span class="notranslate blueTxt" translate="no">Arkist
    	<span title="open artist page" class="tgHid spL">
        	<a href="/artist/4yk4x5w/arkist/index.html" class="">
            	<i class="fa fa-external-link fa-lg linkIcon"></i>
            </a>
        </span>
    </span>
    <span class="" translate="no" class="notranslate"> - </span>
    <span class="blueTxt">
        <span translate="no" class="notranslate">Rendezvous</span>
        <span title="open track page" class="tgHid spL">
            <a href="/track/ls0v7w5/arkist-rendezvous/index.html" title="open track page" translate="no" class="notranslate ">
                <i class="fa fa-external-link fa-lg linkIcon"></i>
            </a>
        </span>
    </span>
</span>

At first I've spent about half an hour trying to select artist's name and track name individually in all of that mess, but then I figured out that I shouldn't overthink it. I only need to scrape the text found inside the trackFormat class to get both artist's and track's name.

I hope that this article will serve as an inspiration of how to connect multiple services together without the need to rely on a service like IFTTT, Zapier or Integromat, as well as what to do once the website you want to connect isn't available in one of these services.

Buy Me a Coffee
Aleksandar Todorović

Published 4 years ago

Mastodon