11  Government Data APIs

TipLearning Objectives

• Manage API keys securely using environment variables and configuration files
• Navigate complex API documentation to identify endpoints, parameters, and variable catalogs
• Query the U.S. Census Bureau API with geographic and variable filters
• Retrieve economic time series from the FRED API with server-side transformations
• Search campaign finance records through the FEC API
• Combine data from multiple government APIs to build richer analytical datasets

TipCompanion Notebook

Run this chapter’s code as you read: download the companion notebook (see Appendix A for all of them).

11.1 Public Data by Mandate

Government APIs exist because of democratic commitments to transparency, not because of business strategy. The Census Bureau is constitutionally mandated to count the population. The Federal Reserve is required to publish economic data. The Federal Election Commission must disclose campaign finance records. As a result, these APIs embody the openness value described in Chapter 3: public data, freely accessible, maintained by public mandate rather than platform goodwill.

These APIs also tend to be more stable than commercial offerings — government agencies do not pivot their business models or get acquired by billionaires. But they are often less polished: the documentation can be dense, variable names can be cryptic, and error messages can be unhelpful. Learning to navigate these tradeoffs is itself a valuable skill.

11.2 API Key Management

Most government APIs require an API key — a unique string that identifies your application, tracks your usage, and enforces rate limits. Proper key management is a professional practice that applies to every authenticated API in this book and beyond.

ImportantGet Your Keys First

This chapter’s tutorial uses three APIs, and each requires its own free key. Request all three now — approval is usually fast, but you cannot run any of the code below without them:

• Census Bureau: https://api.census.gov/data/key_signup.html
• FRED: https://fred.stlouisfed.org/docs/api/api_key.html
• FEC: https://api.open.fec.gov/developers/ (the signup form is at the top of the page)

Store each key as an environment variable, as shown below. For the mechanics of setting environment variables on your operating system, see Missing Manual Chapter 34: Environment Variables and Secrets.

Never hardcode API keys in your notebooks. If you share your notebook (on GitHub, in a class submission, with a collaborator), your key goes with it. Instead, store your keys as environment variables and read them at the top of your notebook:

11.2.1 Environment Variables

import os

# Set in your terminal before launching Jupyter:
#   export CENSUS_API_KEY="your-key-here"     (macOS/Linux)
#   set CENSUS_API_KEY=your-key-here          (Windows)
# ...and likewise for FRED_API_KEY and FEC_API_KEY

census_key = os.environ.get("CENSUS_API_KEY")
fred_key = os.environ.get("FRED_API_KEY")
fec_key = os.environ.get("FEC_API_KEY")

# Fail fast with a helpful message if any key is missing
for name, key in [("CENSUS_API_KEY", census_key),
                  ("FRED_API_KEY", fred_key),
                  ("FEC_API_KEY", fec_key)]:
    if not key:
        raise ValueError(
            f"{name} not found. Sign up for a free key (see the "
            "callout above) and set it as an environment variable "
            "before launching Jupyter."
        )

Every code block in the rest of this chapter assumes these three variables — census_key, fred_key, and fec_key — are defined.

TipAlternative: A JSON Configuration File

Some projects keep keys in a local configuration file instead of environment variables. Create a file called api_keys.json containing {"census": "your-key", "fred": "your-key", "fec": "your-key"}, add it to your .gitignore before your first commit, and load it into the same variable names:

import json

with open("api_keys.json") as f:
    keys = json.load(f)

census_key = keys["census"]
fred_key = keys["fred"]
fec_key = keys["fec"]

Because both approaches end with the same census_key, fred_key, and fec_key variables, every downstream code block in this chapter works unchanged either way.

TipMissing Manual Reference

For a comprehensive guide to API keys, environment variables, .env files, and secrets management, see Missing Manual Chapter 34: Environment Variables and Secrets.

11.3 U.S. Census Bureau API

The Census Bureau provides access to an enormous range of demographic, economic, and housing data. The American Community Survey (ACS) alone contains thousands of variables organized into thematic groups. The challenge is not access — the API is free and well-maintained — but navigation: finding the right variable among thousands.

11.3.1 Navigating the Variable Catalog

The Census API organizes variables into groups. Each group has a code (like B25034 for “Year Structure Built”) and contains multiple variables representing different breakdowns:

import requests
import pandas as pd

# Browse all groups at: https://api.census.gov/data/2022/acs/acs5/groups.html

# Inspect the variables in the "Year Structure Built" group
variables_url = "https://api.census.gov/data/2022/acs/acs5/groups/B25034.json"
response = requests.get(variables_url)
variables = response.json()

for var_id, var_info in sorted(variables["variables"].items()):
    if "label" in var_info and not var_id.endswith("M"):  # Skip margin-of-error
        print(f"{var_id}: {var_info['label'][:80]}")

The variable naming convention follows a pattern: B25034_001E is the total estimate, B25034_002E through B25034_011E are breakdowns by decade of construction, and variables ending in M are margins of error. Learning to read these codes quickly will save you significant time.

11.3.2 Finding the Right Variable

The Census API contains thousands of variables, and finding the right one is often the hardest part of the workflow. The discovery process follows a consistent pattern. Start with the groups index at api.census.gov/data/2022/acs/acs5/groups.html, which lists all thematic groupings. Browse or search for broad categories related to your question — housing, education, income, transportation, language spoken at home.

Once you identify a promising group, drill into its variables. For example, group B01001 (Sex by Age) contains dozens of variables breaking down the population by gender and age bracket. Group B25034 (Year Structure Built) breaks down housing stock by decade of construction. Group B19013 contains median household income. The skill of navigating these catalogs matters far more than memorizing specific variable codes, because the codes change across survey years and table versions.

Use keyword searches to narrow your focus. If you are interested in commuting patterns, search the groups page for “commut” or “transportation” or “travel time.” If you are interested in educational attainment, search for “education” or “school” or “degree.” Multiple groups may be relevant to your question — B15003 covers educational attainment for the population 25 and over, while B14001 covers school enrollment for the population 3 and over. Choosing between them requires reading the variable labels carefully and understanding which population denominator matches your research question.

A practical tip: when you find a variable that looks right, always retrieve it for a geography you know well and sanity-check the values. If you expect Boulder’s median household income to be around $70,000 and the API returns 70000, you are on track. If it returns 7 or 70000000, you may have the wrong variable or the wrong unit. This kind of spot-checking catches errors early and builds confidence in your data pipeline before you scale it to dozens of geographies.

11.3.3 Querying for Specific Places

Census data is organized geographically using FIPS codes. Colorado is state 08; Boulder is place 07850; Longmont is place 45970. You can find FIPS codes at census.gov/library/reference/code-lists.

# Give each API's endpoint its own clearly named variable -- a generic
# name like `url` invites bugs when you move between APIs in one notebook
census_url = "https://api.census.gov/data/2022/acs/acs5"

# Housing age distribution for Boulder, CO
params = {
    "get": "NAME,B25034_001E,B25034_002E,B25034_003E,B25034_004E,B25034_005E",
    "for": "place:07850",
    "in": "state:08",
    "key": census_key
}

response = requests.get(census_url, params=params)
data = response.json()

# First row is column headers, remaining rows are data
df = pd.DataFrame(data[1:], columns=data[0])

# Convert numeric columns from strings
numeric_cols = [c for c in df.columns if c.startswith("B25034")]
df[numeric_cols] = df[numeric_cols].apply(pd.to_numeric)

print(df)

11.3.4 Comparing Multiple Locations

You can pass multiple FIPS codes to compare locations side by side:

# Compare Boulder and Longmont
params["for"] = "place:07850,45970"

response = requests.get(census_url, params=params)
data = response.json()
df_compare = pd.DataFrame(data[1:], columns=data[0])
df_compare[numeric_cols] = df_compare[numeric_cols].apply(pd.to_numeric)

# Reshape for visualization
import matplotlib.pyplot as plt

df_plot = df_compare.set_index("NAME")[numeric_cols].T
df_plot.index = ["Total", "2020+", "2010-2019", "2000-2009", "1990-1999"]

df_plot.plot(kind="bar", figsize=(10, 5))
plt.title("Housing Stock by Decade Built: Boulder vs. Longmont")
plt.ylabel("Number of Housing Units")
plt.xlabel("Decade Built")
plt.xticks(rotation=45)
plt.legend(title="City")
plt.tight_layout()
plt.show()

11.4 FRED API

The Federal Reserve Bank of St. Louis maintains FRED (Federal Reserve Economic Data), arguably the world’s best free source of economic time series. It contains over 800,000 data series covering macroeconomics, banking, labor markets, prices, and more.

11.4.1 Finding Series IDs

FRED organizes data by “series” — each series is a specific time series with a unique ID. The easiest way to find series IDs is to search the FRED website interactively. For example, searching for “median household income Boulder County” will lead you to the series MHICO08013A052NCEN, an annual estimate from the Census Bureau’s Small Area Income and Poverty Estimates (SAIPE) program. Once you know to look, the ID is legible: it embeds the state postal abbreviation (CO) and Boulder County’s FIPS code (08013).

# Retrieve median household income for Boulder County (SAIPE)
fred_url = "https://api.stlouisfed.org/fred/series/observations"

params = {
    "series_id": "MHICO08013A052NCEN",
    "api_key": fred_key,
    "file_type": "json"
}

response = requests.get(fred_url, params=params)
data = response.json()

df_income = pd.DataFrame(data["observations"])
df_income["date"] = pd.to_datetime(df_income["date"])
df_income["value"] = pd.to_numeric(df_income["value"], errors="coerce")

print(df_income.head())

11.4.2 Combining Multiple Series

The real power of FRED emerges when you combine multiple series to tell a richer story:

# Also get CPI data for comparison
params_cpi = {
    "series_id": "CPIAUCSL",  # Consumer Price Index, All Urban Consumers
    "api_key": fred_key,
    "file_type": "json",
    "observation_start": "2000-01-01"
}

response_cpi = requests.get(fred_url, params=params_cpi)
df_cpi = pd.DataFrame(response_cpi.json()["observations"])
df_cpi["date"] = pd.to_datetime(df_cpi["date"])
df_cpi["cpi"] = pd.to_numeric(df_cpi["value"], errors="coerce")

# Merge income and CPI data
df_merged = pd.merge_asof(
    df_income.sort_values("date"),
    df_cpi[["date", "cpi"]].sort_values("date"),
    on="date",
    direction="nearest"
)

11.4.3 Server-Side Transformations

FRED can compute transformations before returning data, saving you work:

# Get CPI as year-over-year percent change
params_transform = {
    "series_id": "CPIAUCSL",
    "api_key": fred_key,
    "file_type": "json",
    "units": "pc1",  # Percent change from 1 year ago
    "observation_start": "2020-01-01"
}

response = requests.get(fred_url, params=params_transform)
df_inflation = pd.DataFrame(response.json()["observations"])
df_inflation["date"] = pd.to_datetime(df_inflation["date"])
df_inflation["inflation_pct"] = pd.to_numeric(df_inflation["value"], errors="coerce")

plt.figure(figsize=(12, 5))
plt.plot(df_inflation["date"], df_inflation["inflation_pct"])
plt.axhline(y=2, color="red", linestyle="--", label="2% target")
plt.title("Year-over-Year Inflation Rate (CPI)")
plt.ylabel("Percent Change")
plt.legend()
plt.tight_layout()
plt.show()

Other available transformations include chg (change from previous value), log (natural log), pca (compounded annual rate of change), and frequency aggregation (converting monthly data to quarterly or annual).

11.4.4 Comparing CPI Across Locations

National economic indicators often mask significant regional variation. The Denver metro area, for instance, experienced housing cost increases that diverged sharply from the national average during the post-pandemic period. FRED provides regional CPI series that let you measure this divergence directly.

Two details matter when you set up this comparison. First, the regional series CUURS49BSA0 is not seasonally adjusted, while the workhorse national series CPIAUCSL is — so for an apples-to-apples year-over-year comparison, use the not-seasonally-adjusted national series CPIAUCNS instead (year-over-year percent changes largely wash out seasonal effects on their own). Second, the BLS publishes the Denver-area CPI bimonthly rather than monthly, so gaps between observations are expected, not a bug in your code.

# Retrieve Denver metro CPI and national CPI
denver_params = {
    "series_id": "CUURS49BSA0",  # CPI, Denver-Aurora-Lakewood (not seasonally adjusted)
    "api_key": fred_key,
    "file_type": "json",
    "units": "pc1",              # Year-over-year percent change
    "observation_start": "2019-01-01"
}

national_params = {
    "series_id": "CPIAUCNS",     # CPI, All Urban Consumers (national, not seasonally adjusted)
    "api_key": fred_key,
    "file_type": "json",
    "units": "pc1",
    "observation_start": "2019-01-01"
}

df_denver = pd.DataFrame(requests.get(fred_url, params=denver_params).json()["observations"])
df_national = pd.DataFrame(requests.get(fred_url, params=national_params).json()["observations"])

df_denver["date"] = pd.to_datetime(df_denver["date"])
df_denver["denver_cpi"] = pd.to_numeric(df_denver["value"], errors="coerce")
df_national["date"] = pd.to_datetime(df_national["date"])
df_national["national_cpi"] = pd.to_numeric(df_national["value"], errors="coerce")

# Merge and plot both series
df_cpi_compare = pd.merge(df_denver[["date", "denver_cpi"]],
                           df_national[["date", "national_cpi"]],
                           on="date", how="inner")

plt.figure(figsize=(12, 5))
plt.plot(df_cpi_compare["date"], df_cpi_compare["denver_cpi"], label="Denver Metro")
plt.plot(df_cpi_compare["date"], df_cpi_compare["national_cpi"], label="National Average")
plt.axhline(y=2, color="gray", linestyle="--", alpha=0.5, label="2% target")
plt.ylabel("Year-over-Year Inflation (%)")
plt.title("Regional vs. National Inflation: Denver Metro Area")
plt.legend()
plt.tight_layout()
plt.show()

What drives local inflation to diverge from national trends? Housing costs are often the primary factor — markets with constrained supply and strong demand (like Denver’s Front Range corridor) see price increases that outpace the national average. Energy prices, labor market tightness, and local policy decisions (such as minimum wage increases or zoning restrictions) all contribute to regional variation. Comparing local and national series is a simple but powerful way to contextualize the economic conditions that affect your community specifically.

11.5 FEC API

The Federal Election Commission provides campaign finance data through a free API. This data enables accountability journalism and research on the influence of money in politics:

# Search for a candidate
fec_search_url = "https://api.open.fec.gov/v1/candidates/search/"
params = {
    "q": "Biden",
    "office": "P",
    "election_year": 2024,
    "api_key": fec_key,
    "per_page": 5
}

response = requests.get(fec_search_url, params=params)
data = response.json()

for candidate in data["results"]:
    print(f"{candidate['name']} ({candidate['party_full']})")
    print(f"  ID: {candidate['candidate_id']}")
    print(f"  State: {candidate['state']}")
    print()

11.5.1 Getting Campaign Finance Data

FEC identifiers follow a convention worth learning: candidate IDs begin with a letter for the office sought (P for president, S for Senate, H for House), while committee IDs begin with C. The ID below is therefore a candidate ID, and this endpoint returns that candidate’s financial totals, aggregated across their authorized committees.

# Get financial totals for a candidate
candidate_id = "P80000722"  # Biden's candidate ID ("P" = presidential)

fec_totals_url = f"https://api.open.fec.gov/v1/candidate/{candidate_id}/totals/"
params = {
    "api_key": fec_key,
    "election_year": 2024
}

response = requests.get(fec_totals_url, params=params)
data = response.json()

for result in data["results"]:
    print(f"Cycle: {result.get('cycle')}")
    print(f"  Receipts: ${result.get('receipts', 0):,.0f}")
    print(f"  Disbursements: ${result.get('disbursements', 0):,.0f}")

11.6 Combining Census and FRED Data

The most compelling analyses often integrate data from multiple government sources to tell a story that no single dataset can tell alone. Census data provides snapshots of demographic and housing characteristics; FRED provides economic time series that capture the conditions under which those characteristics formed. Combining them reveals the relationship between macro-level economic forces and community-level outcomes.

# Define each endpoint explicitly so this block stands on its own --
# reusing a leftover `url` variable from an earlier section is a
# classic source of silent bugs in long notebooks
census_url = "https://api.census.gov/data/2022/acs/acs5"
fred_url = "https://api.stlouisfed.org/fred/series/observations"

# Census: housing vintage by decade for Boulder
housing_params = {
    "get": "NAME,B25034_001E,B25034_002E,B25034_003E,B25034_004E,"
           "B25034_005E,B25034_006E,B25034_007E",
    "for": "place:07850",
    "in": "state:08",
    "key": census_key
}
response = requests.get(census_url, params=housing_params)
housing_data = response.json()

def census_int(value):
    """Convert a Census API value to int, or None if unavailable.

    The Census API reports suppressed or unavailable values with
    sentinel codes such as -666666666 rather than nulls, so never
    trust a bare int() conversion.
    """
    if value is None or str(value).startswith("-6666"):
        return None
    return int(value)

decades = ["2020+", "2010s", "2000s", "1990s", "1980s", "1970s"]
counts = [census_int(housing_data[1][i]) for i in range(2, 8)]

# FRED: 30-year mortgage rates (annual average)
mortgage_params = {
    "series_id": "MORTGAGE30US",
    "api_key": fred_key,
    "file_type": "json",
    "frequency": "a",
    "observation_start": "1970-01-01"
}
df_mortgage = pd.DataFrame(
    requests.get(fred_url, params=mortgage_params).json()["observations"]
)
df_mortgage["date"] = pd.to_datetime(df_mortgage["date"])
df_mortgage["rate"] = pd.to_numeric(df_mortgage["value"], errors="coerce")

# Multi-panel figure
fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(14, 5))

ax1.bar(decades, counts, color="steelblue")
ax1.set_title("Boulder Housing Stock by Decade Built")
ax1.set_ylabel("Housing Units")
ax1.set_xlabel("Decade")

ax2.plot(df_mortgage["date"], df_mortgage["rate"], color="darkred")
ax2.set_title("30-Year Mortgage Rate (Annual Average)")
ax2.set_ylabel("Interest Rate (%)")
plt.tight_layout()
plt.show()

The juxtaposition is revealing. Decades with low mortgage rates tend to correspond to more construction activity, while high-rate periods see building slow down. Boulder’s housing stock tells a specific story: significant construction in the 1970s and 1990s, with constraints imposed by the city’s growth management policies limiting new development regardless of interest rate conditions. This is the value of integrating multiple government data sources — economic trends provide context that makes local patterns interpretable. The Census tells you what the housing stock looks like today; FRED tells you why it looks that way, by revealing the economic conditions that shaped building decisions over decades. Neither dataset alone answers the question as well as both together.

11.7 Exercises

1. Census exploration. Using the Census API, retrieve a variable group of your choice (browse at api.census.gov/data/2022/acs/acs5/groups.html) for at least five geographic areas. Create a comparative visualization and write a brief analytical narrative explaining what the data reveals.

2. FRED time series. Retrieve two related economic time series for your local area or state using the FRED API. Plot them on the same or side-by-side axes. What story does the data tell? Common pairings include housing prices and mortgage rates, unemployment and job creation, or income and cost of living.

3. Campaign finance. Using the FEC API, retrieve contribution data for a local congressional race. How much did each major candidate raise? Can you find differences in the geographic distribution or size distribution of contributions?

4. Multi-source integration. Combine data from at least two different government APIs (Census + FRED, Census + FEC, or FRED + FEC) into a single analysis. For example, compare median household income (Census) with campaign contributions (FEC) across congressional districts, or compare housing construction rates (Census) with interest rates (FRED).

5. County-level analysis. Retrieve a Census variable of your choice for all counties in Colorado using for=county:*&in=state:08. Create a ranked bar chart of the top 20 counties. Write a brief analysis of the geographic patterns you observe.

6. Graduate extension (INFO 5617). Read Lazer et al. (2020) on the obstacles and opportunities facing computational social science. Then design and execute a small study that combines data from at least two of this chapter’s government APIs to examine one obstacle or opportunity the paper raises — for example, the promise and difficulty of linking administrative data across agencies and geographic units. Write a formal limitations section that addresses measurement (what your variables actually capture) and coverage (which people, places, or time periods your linked data misses). The goal is not a definitive finding but a defensible design that takes the paper’s warnings seriously.

NotePublic Interest Connection

Government data APIs embody the value of openness from Chapter 3 — they exist because of democratic commitments to transparency. But government data infrastructure is not immune to erosion. Datasets are occasionally withdrawn, APIs deprecated, and political pressure on statistical agencies can affect what data is collected and how it is presented. The fragility of government data infrastructure is itself a public interest concern.

11.8 Social History and Public Interest

The Census Bureau has been counting the American population since 1790, making the decennial census one of the oldest continuous data collection programs in the world. The American Community Survey, launched in 2005, replaced the census long form with continuous measurement, providing annual estimates of demographic, social, economic, and housing characteristics. The decision to make this data freely accessible through APIs reflects a commitment to the idea that public data should be accessible to all, not just to researchers with institutional subscriptions or specialized software.

FRED was created by the Federal Reserve Bank of St. Louis with a similar philosophy: economic data should be freely accessible to anyone who wants it. Before FRED, accessing Federal Reserve data required specialized terminals or institutional subscriptions. By making the data freely available and well-documented, FRED democratized access to economic information that was previously the province of Wall Street and government insiders.

The FEC’s public disclosure requirements enable some of the most important accountability journalism in American democracy. News organizations, research groups, and citizens use FEC data to track the flow of money in politics, identify potential conflicts of interest, and hold elected officials accountable. This data exists because of the Federal Election Campaign Act (1971) and subsequent legislation mandating transparency in campaign finance.

11.9 Common Issues to Debug

• Census variable codes: The naming convention is opaque. Bookmark the variable catalog and use it as a reference. Variables ending in E are estimates; those ending in M are margins of error.
• FIPS codes: Geographic identifiers like 08 (Colorado) and 07850 (Boulder) are strings, not integers. Do not strip leading zeros.
• FRED series IDs: These are alphanumeric strings; some families follow a pattern (like the SAIPE income series earlier in this chapter), but many do not. Use the FRED website search to find them, then copy the ID into your code exactly.
• API key exposure: The most common mistake. Environment variables never enter your repository; if you use the JSON-file alternative, add api_keys.json to .gitignore before your first commit, or the key will persist in your git history even after deletion.
• Rate limiting: Limits vary by agency. As of mid-2026, the Census API publishes no hard limit for keyed users (but expects polite use), FRED allows roughly 120 requests per minute, and the FEC allows 1,000 requests per hour with an upgraded key available on request. Check each API’s documentation for current figures, and stay well within them, especially during automated collection.

11.10 Key Takeaways

Government APIs provide access to some of the most valuable, reliable, and stable data sources available. The Census Bureau, Federal Reserve, and FEC each serve a specific transparency mandate, and their APIs reflect this mission. The key skills — navigating complex variable catalogs, managing API keys securely, combining data from multiple sources, and understanding the institutional context behind the data — apply to every authenticated API you will encounter. These APIs exist because of public interest mandates, and using them effectively is itself a form of civic engagement.

11.11 Further Reading

• U.S. Census Bureau API: https://www.census.gov/data/developers.html
• FRED API documentation: https://fred.stlouisfed.org/docs/api/fred/
• FEC API documentation: https://api.open.fec.gov/developers/
• Salganik (2018) — Chapter 2: Observing Behavior
• Munzert et al. (2014) — Chapter on working with web APIs

Lazer, David M. J., Alex Pentland, Duncan J. Watts, et al. 2020. “Computational Social Science: Obstacles and Opportunities.” Science 369 (6507): 1060–62. https://doi.org/10.1126/science.aaz8170.

Munzert, Simon, Christian Rubba, Peter Meißner, and Dominic Nyhuis. 2014. Automated Data Collection with R: A Practical Guide to Web Scraping and Text Mining. John Wiley & Sons.

Salganik, Matthew J. 2018. Bit by Bit: Social Research in the Digital Age. Princeton University Press. https://www.bitbybitbook.com/.