12 Social and Media Platform APIs

Learning Objectives

Explain how OAuth-based authentication grants applications scoped access to platform data
Retrieve and analyze subreddit metadata, submissions, comment trees, and user histories from Reddit
Retrieve artist, album, track, and playlist metadata from Spotify
Access decentralized social network data from Bluesky and Mastodon
Compare API design patterns across platforms and assess research implications of different access models

Companion Notebook

Run this chapter’s code as you read: download the companion notebook (see Appendix A for all of them).

12.1 Platforms, Data, and Power

Social and media platform APIs mediate access to an enormous share of public discourse. Unlike the government APIs in Chapter 11 — which exist because of transparency mandates — platform APIs exist at the discretion of private companies. This means access can change overnight, as the Reddit and Twitter cases illustrate.

This chapter teaches authenticated API access through four platforms that represent different points on the spectrum from proprietary to open: Reddit (PRAW), Spotify (spotipy), Bluesky (AT Protocol), and Mastodon (Mastodon.py). Pay attention to the structural differences — they embody the tensions between enclosure and openness discussed in Chapter 3.

Social Data Is People Data

Everything you retrieve in this chapter — posts, comments, profiles, listening patterns — was produced by people, most of whom never imagined their activity in a research dataset. Before collecting, revisit Chapter 2: check the platform’s Terms of Service, consider whether your project needs IRB review, collect the minimum you need, and never gather private or direct-message content. When you report findings, avoid quoting posts verbatim in ways that let a search engine re-identify the author.

12.2 Reddit via PRAW

Reddit provides a free API tier for non-commercial use (100 queries per minute). The PRAW (Python Reddit API Wrapper) library simplifies authentication and data retrieval.

12.2.1 Authentication

Reddit’s API uses OAuth2, the authorization standard behind most platform APIs. The core idea: instead of sending your password with every request, your application exchanges its credentials for a scoped, revocable token. OAuth2 comes in several flows for different situations. The client credentials flow used below authenticates your application and grants read-only access to public data — sufficient for research collection. The authorization code flow additionally asks a user to grant your app permission to act on their behalf (post, vote, read private feeds); you will not need it in this chapter, but you will recognize it from every “Sign in with Google” dialog you have ever clicked through.

# Install: pip install praw
import praw

reddit = praw.Reddit(
    client_id="your_client_id",
    client_secret="your_client_secret",
    user_agent="WebDataScience/1.0 by u/your_username"
)

# Verify the client is working: with app-only credentials, PRAW
# operates in read-only mode
print(f"Read-only mode: {reddit.read_only}")
# Read-only mode: True

You will need to create a Reddit app at https://www.reddit.com/prefs/apps/ and obtain your client ID and secret. Store them in environment variables (or a credentials file excluded from version control), never directly in your notebook.

Missing Manual Reference

For patterns for storing and loading API credentials safely — environment variables, .env files, and what belongs in .gitignore — see Missing Manual Chapter 34: Environment Variables and Secrets.

12.2.2 Exploring Subreddits

import pandas as pd
from datetime import datetime

subreddit = reddit.subreddit("news")

# Subreddit metadata
print(f"Subscribers: {subreddit.subscribers:,}")
print(f"Created: {datetime.fromtimestamp(subreddit.created_utc)}")
print(f"Description: {subreddit.public_description[:200]}")

# Subreddit rules (PRAW returns Rule objects; use attribute access)
for rule in subreddit.rules:
    print(f"- {rule.short_name}")

12.2.3 Retrieving Submissions

# Get the top 25 submissions
submissions = []
for submission in subreddit.top(limit=25, time_filter="week"):
    submissions.append({
        "title": submission.title,
        "score": submission.score,
        "num_comments": submission.num_comments,
        "author": str(submission.author),
        "created_utc": datetime.fromtimestamp(submission.created_utc),
        "url": submission.url
    })

df = pd.DataFrame(submissions)

import matplotlib.pyplot as plt

plt.figure(figsize=(8, 6))
plt.scatter(df["score"], df["num_comments"], alpha=0.7)
plt.xlabel("Score")
plt.ylabel("Number of Comments")
plt.title("r/news: Score vs. Comments (Top 25 Weekly)")
plt.tight_layout()
plt.show()

12.2.4 Comment Trees

# Take the current top submission from the subreddit's hot listing.
# (You can also pass any thread URL from your browser via
# reddit.submission(url="https://www.reddit.com/r/.../comments/..."))
submission = next(subreddit.hot(limit=1))
submission.comments.replace_more(limit=0)  # Flatten the tree

comments = []
for comment in submission.comments.list():
    comments.append({
        "body": comment.body[:200],
        "score": comment.score,
        "depth": comment.depth,
        "author": str(comment.author)
    })

df_comments = pd.DataFrame(comments)
print(f"Total comments: {len(df_comments)}")

12.2.5 Analyzing Comment Engagement

With the comment tree flattened into a DataFrame, you can start asking quantitative questions about how engagement varies with comment structure and position. One natural question: do deeper comments — replies to replies to replies — receive less attention than top-level responses? The depth field PRAW provides makes this straightforward to investigate.

import matplotlib.pyplot as plt

fig, axes = plt.subplots(1, 2, figsize=(12, 5))

# Scatter plot: comment depth vs. score
axes[0].scatter(df_comments["depth"], df_comments["score"], alpha=0.4)
axes[0].set_xlabel("Comment Depth")
axes[0].set_ylabel("Score (Upvotes - Downvotes)")
axes[0].set_title("Comment Depth vs. Score")

# Histogram of comment lengths (word count)
df_comments["word_count"] = df_comments["body"].apply(lambda x: len(x.split()))
axes[1].hist(df_comments["word_count"], bins=30, edgecolor="black", alpha=0.7)
axes[1].set_xlabel("Word Count")
axes[1].set_ylabel("Frequency")
axes[1].set_title("Distribution of Comment Lengths")

plt.tight_layout()
plt.show()

# Summary statistics by depth
depth_summary = df_comments.groupby("depth").agg(
    mean_score=("score", "mean"),
    mean_length=("word_count", "mean"),
    count=("score", "size")
).reset_index()

print(depth_summary)
# depth  mean_score  mean_length  count
# 0        45.2        28.1       312
# 1        12.8        22.5       487
# 2         5.3        18.9       201
# ...

You can also compute the correlation between depth and score to quantify this relationship. A Spearman rank correlation is appropriate here because the score distribution is heavily skewed — a handful of top-level comments accumulate hundreds or thousands of votes while most comments hover near zero.

The pattern you will typically observe is a clear decay: top-level comments (depth 0) receive the highest scores because they are the most visible, appearing directly below the submission. Each additional level of nesting reduces both visibility and engagement. This is not necessarily a reflection of comment quality — it is an artifact of Reddit’s interface design, where nested comments are progressively collapsed and harder to reach.

Comment length tells a complementary story. Most Reddit comments are short — a few sentences at most — but the distribution has a long right tail of substantive, multi-paragraph responses. These longer comments often appear in subreddits with stricter moderation norms (like r/AskHistorians or r/science), where low-effort replies are actively removed. The relationship between platform governance and the data you can collect is not incidental; it is fundamental to interpreting what you find.

Public Interest Connection

Reddit’s 2023 API pricing changes — from free to $0.24 per 1,000 calls — and the shutdown of the Pushshift archive are a live example of enclosure in action. The free tier still supports research use, but the death of third-party apps and Pushshift fundamentally changed what data is available and on whose terms (Baumgartner et al. 2020).

12.3 Spotify via spotipy

Spotify’s API provides metadata about artists, albums, tracks, and playlists — a window into the catalog and popularity dynamics of the world’s largest music streaming platform:

# Install: pip install spotipy
import spotipy
from spotipy.oauth2 import SpotifyClientCredentials

auth = SpotifyClientCredentials(
    client_id="your_spotify_client_id",
    client_secret="your_spotify_client_secret"
)
sp = spotipy.Spotify(auth_manager=auth)

A Shrinking API

In November 2024, Spotify removed several popular endpoints for newly created apps — including audio features, audio analysis, related artists, recommendations, and access to Spotify-owned editorial playlists. Applications registered before the change kept their access; new apps (including yours, if you register one for this course) receive a 403 Forbidden from those endpoints. The examples in this section use only endpoints available to new apps as of mid-2026 — but check the Spotify developer documentation for the current state before building a project around any of them.

12.3.1 Searching and Exploring

# Search for an artist
results = sp.search(q="Queen", type="artist", limit=5)
for artist in results["artists"]["items"]:
    print(f"{artist['name']} — Followers: {artist['followers']['total']:,}")

# Get the top artist's ID
queen_id = results["artists"]["items"][0]["id"]

# Get top tracks
top_tracks = sp.artist_top_tracks(queen_id)
for track in top_tracks["tracks"][:5]:
    print(f"  {track['name']} ({track['album']['name']}) — popularity {track['popularity']}")

Every artist and track carries a popularity score (0–100), Spotify’s algorithmic measure derived from recent play counts. Artists also carry genre labels and follower counts. These fields support a surprising range of research questions about attention, catalog structure, and cultural markets.

12.3.2 Comparing Artists with Catalog Metadata

Metadata becomes revealing when you compare artists. Let us build a profile from three available endpoints — artist metadata, top tracks, and the album catalog — and compare two very different musicians:

def get_artist_profile(sp, artist_name):
    """Build a metadata profile for an artist from public catalog data.

    Parameters
    ----------
    sp : spotipy.Spotify
        Authenticated Spotify client
    artist_name : str
        Name of the artist to search for

    Returns
    -------
    dict
        Artist metadata, top-track popularity, and album release years
    """
    results = sp.search(q=artist_name, type="artist", limit=1)
    artist = results["artists"]["items"][0]

    top_tracks = sp.artist_top_tracks(artist["id"])
    albums = sp.artist_albums(artist["id"], include_groups="album", limit=50)

    return {
        "name": artist["name"],
        "followers": artist["followers"]["total"],
        "artist_popularity": artist["popularity"],
        "genres": artist["genres"],
        "top_track_popularity": [t["popularity"] for t in top_tracks["tracks"]],
        "album_years": sorted({a["release_date"][:4] for a in albums["items"]}),
    }

profile_classical = get_artist_profile(sp, "Yo-Yo Ma")
profile_electronic = get_artist_profile(sp, "Daft Punk")

for p in (profile_classical, profile_electronic):
    print(f"{p['name']}: popularity {p['artist_popularity']}, "
          f"{p['followers']:,} followers, "
          f"albums {p['album_years'][0]}–{p['album_years'][-1]}")

import numpy as np

# Compare the popularity distributions of each artist's top tracks
fig, ax = plt.subplots(figsize=(8, 5))
ax.boxplot(
    [profile_classical["top_track_popularity"],
     profile_electronic["top_track_popularity"]],
    tick_labels=[profile_classical["name"], profile_electronic["name"]]
)
ax.set_ylabel("Track Popularity (0–100)")
ax.set_title("Top-Track Popularity by Artist")
plt.tight_layout()
plt.show()

Even without audio analysis, the catalog tells stories: Daft Punk’s compact discography of studio albums contrasts with Yo-Yo Ma’s decades-long recording career spanning dozens of releases; their popularity scores reveal how streaming-era attention concentrates on some catalogs and spreads thin across others.

For playlist-based questions, sp.playlist_tracks() retrieves the contents of any user-created public playlist (Spotify-owned editorial playlists are no longer accessible to new apps):

# Retrieve tracks from a user-created public playlist by its ID
playlist_id = "spotify_playlist_id_here"  # From the playlist's share link
items = sp.playlist_tracks(playlist_id, limit=100)["items"]

tracks = pd.DataFrame([{
    "name": it["track"]["name"],
    "artist": it["track"]["artists"][0]["name"],
    "popularity": it["track"]["popularity"],
    "duration_min": it["track"]["duration_ms"] / 60000,
    "release_year": int(it["track"]["album"]["release_date"][:4]),
} for it in items if it["track"] is not None])

print(tracks.describe())

12.3.3 The Audio Features Story: An Enclosure Case Study

For more than a decade, the Spotify API’s signature research offering was audio features: algorithmic scores for every track’s danceability, energy, valence (musical positivity), tempo, and more. Hundreds of published studies used them to map genres, model hit prediction, and study the emotional texture of playlists. The November 2024 deprecation cut new applications off from all of it — right as generative AI made music data newly valuable, and without a stated research alternative. It is Chapter 3’s enclosure pattern in miniature: a rich, freely available data resource, withdrawn by unilateral platform decision, with the accumulated literature now resting on measurements that new researchers cannot reproduce.

The episode also carries a methodological lesson that outlives the endpoints. Audio features were never ground truth about music — they were Spotify’s proprietary algorithmic interpretations, learned from training data and reflecting particular cultural assumptions about what makes music “danceable” or “positive.” The same holds for the popularity scores you can still retrieve: they are engagement-weighted, region-influenced, and recomputed on Spotify’s schedule. When you analyze platform data, you are always analyzing the platform’s model of the world as much as the world itself.

12.4 Bluesky via AT Protocol

Bluesky, built on the AT Protocol, represents a structural alternative to proprietary platforms. Its open protocol means that API access is an architectural feature, not a business decision:

# Install: pip install atproto
from atproto import Client

client = Client()
client.login("your-handle.bsky.social", "your-app-password")

# Get your own profile
profile = client.app.bsky.actor.get_profile({"actor": client.me.did})
print(f"Handle: {profile.handle}")
print(f"Followers: {profile.followers_count}")
print(f"Following: {profile.follows_count}")

# Get your timeline
timeline = client.app.bsky.feed.get_timeline({"limit": 10})
for item in timeline.feed:
    post = item.post
    print(f"@{post.author.handle}: {post.record.text[:80]}")

Note the authentication style: an app password generated in Bluesky’s settings, distinct from your account password and revocable at any time — the same delegated-credential principle as OAuth, implemented more simply.

Any account’s public posts are available through get_author_feed:

# Retrieve a user's recent posts (public data; any handle works)
feed = client.app.bsky.feed.get_author_feed(
    {"actor": "bsky.app", "limit": 50}
)

posts = []
for item in feed.feed:
    post = item.post
    posts.append({
        "text": post.record.text,
        "created_at": post.record.created_at,
        "likes": post.like_count,
        "reposts": post.repost_count,
        "is_reply": post.record.reply is not None,
    })

df_posts = pd.DataFrame(posts)
print(f"Posts retrieved: {len(df_posts)}")
print(f"Replies: {df_posts['is_reply'].mean():.0%}")

The key difference from Reddit and Spotify: Bluesky’s protocol is designed so that no single company controls API access. Your identity, social graph, and data are portable across any service that implements the AT Protocol.

12.4.1 Building a Bluesky Social Network

The AT Protocol makes social graph data directly accessible, enabling network analysis that would be difficult on more restricted platforms:

def get_social_graph(client, handle, direction="followers", limit=100):
    """Get followers or following for a Bluesky user.

    Parameters
    ----------
    client : atproto.Client
        Authenticated Bluesky client
    handle : str
        The user's handle (e.g., 'username.bsky.social')
    direction : str
        'followers' or 'following'
    limit : int
        Maximum number of accounts to retrieve

    Returns
    -------
    list
        List of dictionaries with handle and display name
    """
    results = []
    cursor = None

    while len(results) < limit:
        if direction == "followers":
            response = client.app.bsky.graph.get_followers(
                {"actor": handle, "limit": 50, "cursor": cursor}
            )
            accounts = response.followers
        else:
            response = client.app.bsky.graph.get_follows(
                {"actor": handle, "limit": 50, "cursor": cursor}
            )
            accounts = response.follows

        for account in accounts:
            results.append({
                "handle": account.handle,
                "display_name": account.display_name or "",
                "description": (account.description or "")[:100]
            })

        cursor = response.cursor
        if not cursor or not accounts:
            break

    return results[:limit]

# Get followers for a user
followers = get_social_graph(client, "your-handle.bsky.social", "followers")
following = get_social_graph(client, "your-handle.bsky.social", "following")

print(f"Followers: {len(followers)}")
print(f"Following: {len(following)}")

# Find mutual connections
follower_handles = {f["handle"] for f in followers}
following_handles = {f["handle"] for f in following}
mutuals = follower_handles & following_handles
print(f"Mutual connections: {len(mutuals)}")

12.5 Mastodon via Mastodon.py

Mastodon is a decentralized social network built on the ActivityPub protocol. Each server (instance) hosts its own API (Diener and Delcourt 2026):

# Install: pip install Mastodon.py
from mastodon import Mastodon

# One-time setup: register an app on your home instance, then log in.
# Both calls save credentials to local files you should .gitignore.
# Mastodon.create_app(
#     "WebDataScience",
#     api_base_url="https://mastodon.social",
#     to_file="app_cred.secret"
# )
# Mastodon(client_id="app_cred.secret").log_in(
#     "your-email", "your-password", to_file="user_cred.secret"
# )

# In every session afterward, authenticate with the saved token
mastodon = Mastodon(
    access_token="user_cred.secret",
    api_base_url="https://mastodon.social"
)

# Get the public timeline
from bs4 import BeautifulSoup

timeline = mastodon.timeline_public(limit=10)
for toot in timeline:
    # Statuses arrive as HTML; strip tags before analysis
    text = BeautifulSoup(toot["content"], "html.parser").get_text()
    print(f"@{toot['account']['username']}: {text[:80]}")

12.5.1 Working Across Instances

A key difference between Mastodon and centralized platforms is that there is no single API endpoint for “all of Mastodon.” Each instance maintains its own API, and you can only directly access the data on instances you authenticate with. This federated architecture means that comprehensive data collection requires accessing multiple instances — a fundamentally different research design challenge:

# Search for accounts across the instance you're connected to
results = mastodon.account_search("data science", limit=10)
for account in results:
    print(f"@{account['acct']} — {account['followers_count']} followers")
    print(f"  {account['note'][:100]}")
    print()

# Get a user's recent statuses (toots)
user_id = results[0]["id"]
statuses = mastodon.account_statuses(user_id, limit=20)

for status in statuses[:5]:
    # created_at is a datetime object; content is HTML
    text = BeautifulSoup(status["content"], "html.parser").get_text()
    print(f"  [{status['created_at'].date()}] {text[:100]}")

# Track a hashtag
tagged = mastodon.timeline_hashtag("datascience", limit=20)
print(f"\nRecent posts tagged #datascience: {len(tagged)}")

12.6 Comparing Platform Architectures

The four platforms in this chapter represent fundamentally different governance architectures, and these differences shape what data is available, on what terms, and how durable the access is.

Platform	Protocol	Authentication	Rate Limit (free)	Data Ownership
Reddit	Proprietary REST API	OAuth2	100 QPM	Platform-controlled
Spotify	Proprietary REST API	OAuth2 / Client Credentials	~180 RPM	Platform-controlled
Bluesky	AT Protocol (open)	App password	Generous, per-PDS	User-portable
Mastodon	ActivityPub (open)	OAuth2, per-instance	Varies by instance	Instance-governed

The structural difference matters: Reddit and Spotify can unilaterally change their API terms because they control both the platform and the protocol. Bluesky and Mastodon cannot — or at least, no single entity can — because the protocols are open standards that anyone can implement. This is the architectural embodiment of the ownership value from Chapter 3.

12.7 Exercises

Reddit analysis. Compare two subreddits on the same broad topic (e.g., r/science and r/askscience). How do submission patterns, comment engagement, and moderation structures differ?
Discography metadata. Using spotipy, analyze an artist’s full discography with sp.artist_albums() and sp.album_tracks(). How has their release cadence changed over their career? Do older or newer releases hold higher popularity scores today? What does that pattern suggest about how streaming shapes attention to back catalogs?
Open vs. closed. Attempt to retrieve the same type of information (user profile, social graph, recent posts) from both Bluesky and Reddit. Compare the ease of access, the richness of the data, and the authentication requirements. What does this comparison tell you about platform governance?
Playlist fingerprints. Choose two user-created public Spotify playlists representing different genres (e.g., country vs. electronic — note that Spotify-owned editorial playlists are not accessible to new apps). Retrieve their contents with sp.playlist_tracks() and compare the playlists on available metadata: track popularity, duration, release year, and the number of distinct artists. Create overlapping distribution plots for at least two of these measures. How well can you distinguish the genres from metadata alone — and what would you be missing without audio analysis?
Bluesky analysis. Retrieve the 50 most recent posts from a Bluesky account of your choice using get_author_feed (demonstrated in this chapter). Compute summary statistics: average post length, posting frequency (posts per day), fraction of posts that are replies vs. original posts, and the most common words used. How does this activity profile compare to what you might expect from the same account on Twitter/X? What limitations of the AT Protocol API did you encounter during data collection?
Graduate extension (INFO 5617). Read Bruns (2019) and Davidson et al. (2023). Then design (do not execute) a measurement plan for studying the same construct — for example, cross-partisan interaction, or the spread of health misinformation — on one proprietary platform and one federated platform from this chapter. For each: specify the endpoints and fields you would use, the sampling frame, what the platform’s access rules prevent you from observing, and how those constraints could bias your conclusions. Conclude with an assessment of whether the two platforms’ results would be comparable at all.

12.8 Social History and Public Interest

The contrast between Reddit, Spotify, Bluesky, and Mastodon illustrates the governance spectrum described in Chapter 3. Reddit’s 2023 changes demonstrate how a platform can unilaterally restrict access that researchers depended on. Spotify’s trajectory — rich audio analysis data for a decade, then a 2024 deprecation that cut off new research applications — shows that even “productive commercial arrangements” narrow when the business calculus shifts. Bluesky and Mastodon represent structural alternatives where open protocols make enclosure architecturally harder.

The PRAW library and Mastodon.py continue to work because their platforms maintain documented APIs. But the history of Twitter’s API — which went from the most important data source in computational social science to essentially inaccessible within a year of its acquisition — is a reminder that platform goodwill is not infrastructure.

The Fediverse offers two distinct architectural responses to this problem. ActivityPub, the protocol underlying Mastodon, creates a federated network where each instance is independently operated but can communicate with others — much like email servers. No single company controls the protocol, so no single company can revoke access. The AT Protocol, which powers Bluesky, takes a different approach centered on portable identity: your account data, social graph, and content can move between providers, preventing the lock-in that makes platform migration so costly. Both represent structural alternatives to centralized platforms, distributing control rather than concentrating it. This is the architectural embodiment of ownership (Chapter 3) — users and communities, not corporations, control the infrastructure. For researchers, these open protocols mean that data access is a design feature rather than a business concession, offering more durable foundations for longitudinal research than any corporate API ever could. The trade-off is fragmentation: collecting comprehensive data from a federated network requires engaging with multiple instances, each with its own norms, moderation policies, and technical configurations. Decentralization solves the single-point-of-failure problem but introduces new challenges for research design that centralized platforms never posed.

12.9 Common Issues to Debug

PRAW OAuthException: Double-check your client_id, client_secret, and user_agent. The user_agent must be unique and descriptive.
Spotify 403 Forbidden on audio features, recommendations, or related artists: These endpoints were deprecated for apps created after November 2024. This is not a bug in your code — the access no longer exists for new apps.
Spotify client credentials vs. authorization code: Client credentials flow cannot access user-specific data like listening history. It works for public search, artist, album, track, and user-created playlist data.
Bluesky atproto SDK: Still evolving; API may change between versions. Check the changelog before upgrading.
Mastodon instance differences: Different instances have different rate limits, rules, and may block automated access. Always register your app properly.
Mastodon HTML content: Status content fields arrive as HTML and created_at fields as datetime objects — strip tags with BeautifulSoup and use datetime methods rather than string slicing.

12.10 Key Takeaways

Platform APIs provide access to behavioral data at unprecedented scale, but that access is conditional, costly, and fragile. The pattern across all four platforms is the same: authenticate, construct requests, parse nested JSON, convert to DataFrames. The difference is in governance: who controls the data, on what terms, and what happens when they change the rules. Build skills across both proprietary and open platforms so that the closure of any single source does not shut down your research.

12.11 Further Reading

PRAW documentation: https://praw.readthedocs.io/
spotipy documentation: https://spotipy.readthedocs.io/
AT Protocol SDK: https://atproto.blue/
Mastodon.py documentation: https://mastodonpy.readthedocs.io/
Baumgartner et al. (2020) — the Pushshift Reddit dataset
Diener and Delcourt (2026) — the Mastodon.py library
Bruns (2019) — platforms versus scholarly research after the APIcalypse

Baumgartner, Jason, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. “The Pushshift Reddit Dataset.” Proceedings of the International AAAI Conference on Web and Social Media 14: 830–39.

Bruns, Axel. 2019. “After the ’APIcalypse’: Social Media Platforms and Their Fight Against Critical Scholarly Research.” Information, Communication & Society 22 (11): 1544–66. https://doi.org/10.1080/1369118X.2019.1637447.

Davidson, Brittany I., Darja Wischerath, Daniel Racek, et al. 2023. “Platform-Controlled Social Media APIs Threaten Open Science.” Nature Human Behaviour 7: 2054–57. https://doi.org/10.1038/s41562-023-01750-2.

Diener, Lorenz, and Corentin Delcourt. 2026. “Mastodon.py: A Python Library for the Mastodon API.” Journal of Open Source Software 11 (120): 8946. https://doi.org/10.21105/joss.08946.

# Social and Media Platform APIs {#sec-social} ::: {.callout-tip} ## Learning Objectives - Explain how OAuth-based authentication grants applications scoped access to platform data - Retrieve and analyze subreddit metadata, submissions, comment trees, and user histories from Reddit - Retrieve artist, album, track, and playlist metadata from Spotify - Access decentralized social network data from Bluesky and Mastodon - Compare API design patterns across platforms and assess research implications of different access models ::: ::: {.callout-tip} ## Companion Notebook Run this chapter's code as you read: download the [companion notebook](notebooks/ch-12-social.ipynb) (see @sec-appendix-notebooks for all of them). ::: ## Platforms, Data, and Power Social and media platform APIs mediate access to an enormous share of public discourse. Unlike the government APIs in @sec-government — which exist because of transparency mandates — platform APIs exist at the discretion of private companies. This means access can change overnight, as the Reddit and Twitter cases illustrate. This chapter teaches authenticated API access through four platforms that represent different points on the spectrum from proprietary to open: Reddit (PRAW), Spotify (spotipy), Bluesky (AT Protocol), and Mastodon (Mastodon.py). Pay attention to the structural differences — they embody the tensions between enclosure and openness discussed in @sec-post-api. ::: {.callout-warning} ## Social Data Is People Data Everything you retrieve in this chapter — posts, comments, profiles, listening patterns — was produced by people, most of whom never imagined their activity in a research dataset. Before collecting, revisit @sec-ethics: check the platform's Terms of Service, consider whether your project needs IRB review, collect the minimum you need, and never gather private or direct-message content. When you report findings, avoid quoting posts verbatim in ways that let a search engine re-identify the author. ::: ## Reddit via PRAW Reddit provides a free API tier for non-commercial use (100 queries per minute). The PRAW (Python Reddit API Wrapper) library simplifies authentication and data retrieval. ### Authentication Reddit's API uses **OAuth2**, the authorization standard behind most platform APIs. The core idea: instead of sending your password with every request, your application exchanges its credentials for a scoped, revocable token. OAuth2 comes in several *flows* for different situations. The **client credentials** flow used below authenticates your *application* and grants read-only access to public data — sufficient for research collection. The **authorization code** flow additionally asks a *user* to grant your app permission to act on their behalf (post, vote, read private feeds); you will not need it in this chapter, but you will recognize it from every "Sign in with Google" dialog you have ever clicked through. ```{python} # Install: pip install praw import praw reddit = praw.Reddit( client_id="your_client_id", client_secret="your_client_secret", user_agent="WebDataScience/1.0 by u/your_username" ) # Verify the client is working: with app-only credentials, PRAW # operates in read-only mode print(f"Read-only mode: {reddit.read_only}") # Read-only mode: True ``` You will need to create a Reddit app at <https://www.reddit.com/prefs/apps/> and obtain your client ID and secret. Store them in environment variables (or a credentials file excluded from version control), never directly in your notebook. ::: {.callout-tip} ## Missing Manual Reference For patterns for storing and loading API credentials safely — environment variables, `.env` files, and what belongs in `.gitignore` — see *Missing Manual* Chapter 34: Environment Variables and Secrets. ::: ### Exploring Subreddits ```{python} import pandas as pd from datetime import datetime subreddit = reddit.subreddit("news") # Subreddit metadata print(f"Subscribers: {subreddit.subscribers:,}") print(f"Created: {datetime.fromtimestamp(subreddit.created_utc)}") print(f"Description: {subreddit.public_description[:200]}") # Subreddit rules (PRAW returns Rule objects; use attribute access) for rule in subreddit.rules: print(f"- {rule.short_name}") ``` ### Retrieving Submissions ```{python} # Get the top 25 submissions submissions = [] for submission in subreddit.top(limit=25, time_filter="week"): submissions.append({ "title": submission.title, "score": submission.score, "num_comments": submission.num_comments, "author": str(submission.author), "created_utc": datetime.fromtimestamp(submission.created_utc), "url": submission.url }) df = pd.DataFrame(submissions) import matplotlib.pyplot as plt plt.figure(figsize=(8, 6)) plt.scatter(df["score"], df["num_comments"], alpha=0.7) plt.xlabel("Score") plt.ylabel("Number of Comments") plt.title("r/news: Score vs. Comments (Top 25 Weekly)") plt.tight_layout() plt.show() ``` ### Comment Trees ```{python} # Take the current top submission from the subreddit's hot listing. # (You can also pass any thread URL from your browser via # reddit.submission(url="https://www.reddit.com/r/.../comments/...")) submission = next(subreddit.hot(limit=1)) submission.comments.replace_more(limit=0) # Flatten the tree comments = [] for comment in submission.comments.list(): comments.append({ "body": comment.body[:200], "score": comment.score, "depth": comment.depth, "author": str(comment.author) }) df_comments = pd.DataFrame(comments) print(f"Total comments: {len(df_comments)}") ``` ### Analyzing Comment Engagement With the comment tree flattened into a DataFrame, you can start asking quantitative questions about how engagement varies with comment structure and position. One natural question: do deeper comments — replies to replies to replies — receive less attention than top-level responses? The `depth` field PRAW provides makes this straightforward to investigate. ```{python} import matplotlib.pyplot as plt fig, axes = plt.subplots(1, 2, figsize=(12, 5)) # Scatter plot: comment depth vs. score axes[0].scatter(df_comments["depth"], df_comments["score"], alpha=0.4) axes[0].set_xlabel("Comment Depth") axes[0].set_ylabel("Score (Upvotes - Downvotes)") axes[0].set_title("Comment Depth vs. Score") # Histogram of comment lengths (word count) df_comments["word_count"] = df_comments["body"].apply(lambda x: len(x.split())) axes[1].hist(df_comments["word_count"], bins=30, edgecolor="black", alpha=0.7) axes[1].set_xlabel("Word Count") axes[1].set_ylabel("Frequency") axes[1].set_title("Distribution of Comment Lengths") plt.tight_layout() plt.show() ``` ```{python} # Summary statistics by depth depth_summary = df_comments.groupby("depth").agg( mean_score=("score", "mean"), mean_length=("word_count", "mean"), count=("score", "size") ).reset_index() print(depth_summary) # depth mean_score mean_length count # 0 45.2 28.1 312 # 1 12.8 22.5 487 # 2 5.3 18.9 201 # ... ``` You can also compute the correlation between depth and score to quantify this relationship. A Spearman rank correlation is appropriate here because the score distribution is heavily skewed — a handful of top-level comments accumulate hundreds or thousands of votes while most comments hover near zero. The pattern you will typically observe is a clear decay: top-level comments (depth 0) receive the highest scores because they are the most visible, appearing directly below the submission. Each additional level of nesting reduces both visibility and engagement. This is not necessarily a reflection of comment quality — it is an artifact of Reddit's interface design, where nested comments are progressively collapsed and harder to reach. Comment length tells a complementary story. Most Reddit comments are short — a few sentences at most — but the distribution has a long right tail of substantive, multi-paragraph responses. These longer comments often appear in subreddits with stricter moderation norms (like r/AskHistorians or r/science), where low-effort replies are actively removed. The relationship between platform governance and the data you can collect is not incidental; it is fundamental to interpreting what you find. ::: {.callout-note} ## Public Interest Connection Reddit's 2023 API pricing changes — from free to $0.24 per 1,000 calls — and the shutdown of the Pushshift archive are a live example of **enclosure** in action. The free tier still supports research use, but the death of third-party apps and Pushshift fundamentally changed what data is available and on whose terms [@baumgartner2020pushshift]. ::: ## Spotify via spotipy Spotify's API provides metadata about artists, albums, tracks, and playlists — a window into the catalog and popularity dynamics of the world's largest music streaming platform: ```{python} # Install: pip install spotipy import spotipy from spotipy.oauth2 import SpotifyClientCredentials auth = SpotifyClientCredentials( client_id="your_spotify_client_id", client_secret="your_spotify_client_secret" ) sp = spotipy.Spotify(auth_manager=auth) ``` ::: {.callout-warning} ## A Shrinking API In November 2024, Spotify removed several popular endpoints for newly created apps — including audio features, audio analysis, related artists, recommendations, and access to Spotify-owned editorial playlists. Applications registered before the change kept their access; new apps (including yours, if you register one for this course) receive a `403 Forbidden` from those endpoints. The examples in this section use only endpoints available to new apps as of mid-2026 — but check the [Spotify developer documentation](https://developer.spotify.com/documentation/web-api) for the current state before building a project around any of them. ::: ### Searching and Exploring ```{python} # Search for an artist results = sp.search(q="Queen", type="artist", limit=5) for artist in results["artists"]["items"]: print(f"{artist['name']} — Followers: {artist['followers']['total']:,}") # Get the top artist's ID queen_id = results["artists"]["items"][0]["id"] # Get top tracks top_tracks = sp.artist_top_tracks(queen_id) for track in top_tracks["tracks"][:5]: print(f" {track['name']} ({track['album']['name']}) — popularity {track['popularity']}") ``` Every artist and track carries a **popularity** score (0–100), Spotify's algorithmic measure derived from recent play counts. Artists also carry genre labels and follower counts. These fields support a surprising range of research questions about attention, catalog structure, and cultural markets. ### Comparing Artists with Catalog Metadata Metadata becomes revealing when you compare artists. Let us build a profile from three available endpoints — artist metadata, top tracks, and the album catalog — and compare two very different musicians: ```{python} def get_artist_profile(sp, artist_name): """Build a metadata profile for an artist from public catalog data. Parameters ---------- sp : spotipy.Spotify Authenticated Spotify client artist_name : str Name of the artist to search for Returns ------- dict Artist metadata, top-track popularity, and album release years """ results = sp.search(q=artist_name, type="artist", limit=1) artist = results["artists"]["items"][0] top_tracks = sp.artist_top_tracks(artist["id"]) albums = sp.artist_albums(artist["id"], include_groups="album", limit=50) return { "name": artist["name"], "followers": artist["followers"]["total"], "artist_popularity": artist["popularity"], "genres": artist["genres"], "top_track_popularity": [t["popularity"] for t in top_tracks["tracks"]], "album_years": sorted({a["release_date"][:4] for a in albums["items"]}), } profile_classical = get_artist_profile(sp, "Yo-Yo Ma") profile_electronic = get_artist_profile(sp, "Daft Punk") for p in (profile_classical, profile_electronic): print(f"{p['name']}: popularity {p['artist_popularity']}, " f"{p['followers']:,} followers, " f"albums {p['album_years'][0]}–{p['album_years'][-1]}") ``` ```{python} import numpy as np # Compare the popularity distributions of each artist's top tracks fig, ax = plt.subplots(figsize=(8, 5)) ax.boxplot( [profile_classical["top_track_popularity"], profile_electronic["top_track_popularity"]], tick_labels=[profile_classical["name"], profile_electronic["name"]] ) ax.set_ylabel("Track Popularity (0–100)") ax.set_title("Top-Track Popularity by Artist") plt.tight_layout() plt.show() ``` Even without audio analysis, the catalog tells stories: Daft Punk's compact discography of studio albums contrasts with Yo-Yo Ma's decades-long recording career spanning dozens of releases; their popularity scores reveal how streaming-era attention concentrates on some catalogs and spreads thin across others. For playlist-based questions, `sp.playlist_tracks()` retrieves the contents of any *user-created* public playlist (Spotify-owned editorial playlists are no longer accessible to new apps): ```{python} # Retrieve tracks from a user-created public playlist by its ID playlist_id = "spotify_playlist_id_here" # From the playlist's share link items = sp.playlist_tracks(playlist_id, limit=100)["items"] tracks = pd.DataFrame([{ "name": it["track"]["name"], "artist": it["track"]["artists"][0]["name"], "popularity": it["track"]["popularity"], "duration_min": it["track"]["duration_ms"] / 60000, "release_year": int(it["track"]["album"]["release_date"][:4]), } for it in items if it["track"] is not None]) print(tracks.describe()) ``` ### The Audio Features Story: An Enclosure Case Study For more than a decade, the Spotify API's signature research offering was **audio features**: algorithmic scores for every track's danceability, energy, valence (musical positivity), tempo, and more. Hundreds of published studies used them to map genres, model hit prediction, and study the emotional texture of playlists. The November 2024 deprecation cut new applications off from all of it — right as generative AI made music data newly valuable, and without a stated research alternative. It is @sec-post-api's **enclosure** pattern in miniature: a rich, freely available data resource, withdrawn by unilateral platform decision, with the accumulated literature now resting on measurements that new researchers cannot reproduce. The episode also carries a methodological lesson that outlives the endpoints. Audio features were never ground truth about music — they were Spotify's proprietary algorithmic interpretations, learned from training data and reflecting particular cultural assumptions about what makes music "danceable" or "positive." The same holds for the popularity scores you *can* still retrieve: they are engagement-weighted, region-influenced, and recomputed on Spotify's schedule. When you analyze platform data, you are always analyzing the platform's model of the world as much as the world itself. ## Bluesky via AT Protocol Bluesky, built on the AT Protocol, represents a structural alternative to proprietary platforms. Its open protocol means that API access is an architectural feature, not a business decision: ```{python} # Install: pip install atproto from atproto import Client client = Client() client.login("your-handle.bsky.social", "your-app-password") # Get your own profile profile = client.app.bsky.actor.get_profile({"actor": client.me.did}) print(f"Handle: {profile.handle}") print(f"Followers: {profile.followers_count}") print(f"Following: {profile.follows_count}") # Get your timeline timeline = client.app.bsky.feed.get_timeline({"limit": 10}) for item in timeline.feed: post = item.post print(f"@{post.author.handle}: {post.record.text[:80]}") ``` Note the authentication style: an **app password** generated in Bluesky's settings, distinct from your account password and revocable at any time — the same delegated-credential principle as OAuth, implemented more simply. Any account's public posts are available through `get_author_feed`: ```{python} # Retrieve a user's recent posts (public data; any handle works) feed = client.app.bsky.feed.get_author_feed( {"actor": "bsky.app", "limit": 50} ) posts = [] for item in feed.feed: post = item.post posts.append({ "text": post.record.text, "created_at": post.record.created_at, "likes": post.like_count, "reposts": post.repost_count, "is_reply": post.record.reply is not None, }) df_posts = pd.DataFrame(posts) print(f"Posts retrieved: {len(df_posts)}") print(f"Replies: {df_posts['is_reply'].mean():.0%}") ``` The key difference from Reddit and Spotify: Bluesky's protocol is designed so that no single company controls API access. Your identity, social graph, and data are portable across any service that implements the AT Protocol. ### Building a Bluesky Social Network The AT Protocol makes social graph data directly accessible, enabling network analysis that would be difficult on more restricted platforms: ```{python} def get_social_graph(client, handle, direction="followers", limit=100): """Get followers or following for a Bluesky user. Parameters ---------- client : atproto.Client Authenticated Bluesky client handle : str The user's handle (e.g., 'username.bsky.social') direction : str 'followers' or 'following' limit : int Maximum number of accounts to retrieve Returns ------- list List of dictionaries with handle and display name """ results = [] cursor = None while len(results) < limit: if direction == "followers": response = client.app.bsky.graph.get_followers( {"actor": handle, "limit": 50, "cursor": cursor} ) accounts = response.followers else: response = client.app.bsky.graph.get_follows( {"actor": handle, "limit": 50, "cursor": cursor} ) accounts = response.follows for account in accounts: results.append({ "handle": account.handle, "display_name": account.display_name or "", "description": (account.description or "")[:100] }) cursor = response.cursor if not cursor or not accounts: break return results[:limit] # Get followers for a user followers = get_social_graph(client, "your-handle.bsky.social", "followers") following = get_social_graph(client, "your-handle.bsky.social", "following") print(f"Followers: {len(followers)}") print(f"Following: {len(following)}") # Find mutual connections follower_handles = {f["handle"] for f in followers} following_handles = {f["handle"] for f in following} mutuals = follower_handles & following_handles print(f"Mutual connections: {len(mutuals)}") ``` ## Mastodon via Mastodon.py Mastodon is a decentralized social network built on the ActivityPub protocol. Each server (instance) hosts its own API [@diener2026mastodonpy]: ```{python} # Install: pip install Mastodon.py from mastodon import Mastodon # One-time setup: register an app on your home instance, then log in. # Both calls save credentials to local files you should .gitignore. # Mastodon.create_app( # "WebDataScience", # api_base_url="https://mastodon.social", # to_file="app_cred.secret" # ) # Mastodon(client_id="app_cred.secret").log_in( # "your-email", "your-password", to_file="user_cred.secret" # ) # In every session afterward, authenticate with the saved token mastodon = Mastodon( access_token="user_cred.secret", api_base_url="https://mastodon.social" ) # Get the public timeline from bs4 import BeautifulSoup timeline = mastodon.timeline_public(limit=10) for toot in timeline: # Statuses arrive as HTML; strip tags before analysis text = BeautifulSoup(toot["content"], "html.parser").get_text() print(f"@{toot['account']['username']}: {text[:80]}") ``` ### Working Across Instances A key difference between Mastodon and centralized platforms is that there is no single API endpoint for "all of Mastodon." Each instance maintains its own API, and you can only directly access the data on instances you authenticate with. This federated architecture means that comprehensive data collection requires accessing multiple instances — a fundamentally different research design challenge: ```{python} # Search for accounts across the instance you're connected to results = mastodon.account_search("data science", limit=10) for account in results: print(f"@{account['acct']} — {account['followers_count']} followers") print(f" {account['note'][:100]}") print() # Get a user's recent statuses (toots) user_id = results[0]["id"] statuses = mastodon.account_statuses(user_id, limit=20) for status in statuses[:5]: # created_at is a datetime object; content is HTML text = BeautifulSoup(status["content"], "html.parser").get_text() print(f" [{status['created_at'].date()}] {text[:100]}") # Track a hashtag tagged = mastodon.timeline_hashtag("datascience", limit=20) print(f"\nRecent posts tagged #datascience: {len(tagged)}") ``` ## Comparing Platform Architectures The four platforms in this chapter represent fundamentally different governance architectures, and these differences shape what data is available, on what terms, and how durable the access is. | Platform | Protocol | Authentication | Rate Limit (free) | Data Ownership | |----------|----------|----------------|-------------------|----------------| | Reddit | Proprietary REST API | OAuth2 | 100 QPM | Platform-controlled | | Spotify | Proprietary REST API | OAuth2 / Client Credentials | ~180 RPM | Platform-controlled | | Bluesky | AT Protocol (open) | App password | Generous, per-PDS | User-portable | | Mastodon | ActivityPub (open) | OAuth2, per-instance | Varies by instance | Instance-governed | The structural difference matters: Reddit and Spotify can unilaterally change their API terms because they control both the platform and the protocol. Bluesky and Mastodon cannot — or at least, no single entity can — because the protocols are open standards that anyone can implement. This is the architectural embodiment of the **ownership** value from @sec-post-api. ## Exercises 1. **Reddit analysis.** Compare two subreddits on the same broad topic (e.g., r/science and r/askscience). How do submission patterns, comment engagement, and moderation structures differ? 2. **Discography metadata.** Using spotipy, analyze an artist's full discography with `sp.artist_albums()` and `sp.album_tracks()`. How has their release cadence changed over their career? Do older or newer releases hold higher popularity scores today? What does that pattern suggest about how streaming shapes attention to back catalogs? 3. **Open vs. closed.** Attempt to retrieve the same type of information (user profile, social graph, recent posts) from both Bluesky and Reddit. Compare the ease of access, the richness of the data, and the authentication requirements. What does this comparison tell you about platform governance? 4. **Playlist fingerprints.** Choose two user-created public Spotify playlists representing different genres (e.g., country vs. electronic — note that Spotify-owned editorial playlists are not accessible to new apps). Retrieve their contents with `sp.playlist_tracks()` and compare the playlists on available metadata: track popularity, duration, release year, and the number of distinct artists. Create overlapping distribution plots for at least two of these measures. How well can you distinguish the genres from metadata alone — and what would you be missing without audio analysis? 5. **Bluesky analysis.** Retrieve the 50 most recent posts from a Bluesky account of your choice using `get_author_feed` (demonstrated in this chapter). Compute summary statistics: average post length, posting frequency (posts per day), fraction of posts that are replies vs. original posts, and the most common words used. How does this activity profile compare to what you might expect from the same account on Twitter/X? What limitations of the AT Protocol API did you encounter during data collection? 6. **Graduate extension (INFO 5617).** Read @bruns2019after and @davidson2023platform. Then design (do not execute) a measurement plan for studying the same construct — for example, cross-partisan interaction, or the spread of health misinformation — on one proprietary platform and one federated platform from this chapter. For each: specify the endpoints and fields you would use, the sampling frame, what the platform's access rules prevent you from observing, and how those constraints could bias your conclusions. Conclude with an assessment of whether the two platforms' results would be comparable at all. ## Social History and Public Interest The contrast between Reddit, Spotify, Bluesky, and Mastodon illustrates the governance spectrum described in @sec-post-api. Reddit's 2023 changes demonstrate how a platform can unilaterally restrict access that researchers depended on. Spotify's trajectory — rich audio analysis data for a decade, then a 2024 deprecation that cut off new research applications — shows that even "productive commercial arrangements" narrow when the business calculus shifts. Bluesky and Mastodon represent structural alternatives where open protocols make enclosure architecturally harder. The PRAW library and Mastodon.py continue to work because their platforms maintain documented APIs. But the history of Twitter's API — which went from the most important data source in computational social science to essentially inaccessible within a year of its acquisition — is a reminder that platform goodwill is not infrastructure. The Fediverse offers two distinct architectural responses to this problem. ActivityPub, the protocol underlying Mastodon, creates a federated network where each instance is independently operated but can communicate with others — much like email servers. No single company controls the protocol, so no single company can revoke access. The AT Protocol, which powers Bluesky, takes a different approach centered on portable identity: your account data, social graph, and content can move between providers, preventing the lock-in that makes platform migration so costly. Both represent structural alternatives to centralized platforms, distributing control rather than concentrating it. This is the architectural embodiment of **ownership** (@sec-post-api) — users and communities, not corporations, control the infrastructure. For researchers, these open protocols mean that data access is a design feature rather than a business concession, offering more durable foundations for longitudinal research than any corporate API ever could. The trade-off is fragmentation: collecting comprehensive data from a federated network requires engaging with multiple instances, each with its own norms, moderation policies, and technical configurations. Decentralization solves the single-point-of-failure problem but introduces new challenges for research design that centralized platforms never posed. ## Common Issues to Debug - **PRAW `OAuthException`**: Double-check your `client_id`, `client_secret`, and `user_agent`. The user_agent must be unique and descriptive. - **Spotify `403 Forbidden` on audio features, recommendations, or related artists**: These endpoints were deprecated for apps created after November 2024. This is not a bug in your code — the access no longer exists for new apps. - **Spotify client credentials vs. authorization code**: Client credentials flow cannot access user-specific data like listening history. It works for public search, artist, album, track, and user-created playlist data. - **Bluesky `atproto` SDK**: Still evolving; API may change between versions. Check the changelog before upgrading. - **Mastodon instance differences**: Different instances have different rate limits, rules, and may block automated access. Always register your app properly. - **Mastodon HTML content**: Status `content` fields arrive as HTML and `created_at` fields as `datetime` objects — strip tags with BeautifulSoup and use datetime methods rather than string slicing. ## Key Takeaways Platform APIs provide access to behavioral data at unprecedented scale, but that access is conditional, costly, and fragile. The pattern across all four platforms is the same: authenticate, construct requests, parse nested JSON, convert to DataFrames. The difference is in governance: who controls the data, on what terms, and what happens when they change the rules. Build skills across both proprietary and open platforms so that the closure of any single source does not shut down your research. ## Further Reading - PRAW documentation: <https://praw.readthedocs.io/> - spotipy documentation: <https://spotipy.readthedocs.io/> - AT Protocol SDK: <https://atproto.blue/> - Mastodon.py documentation: <https://mastodonpy.readthedocs.io/> - @baumgartner2020pushshift — the Pushshift Reddit dataset - @diener2026mastodonpy — the Mastodon.py library - @bruns2019after — platforms versus scholarly research after the APIcalypse