2  Documentation, Markdown, and Errors

2.1 Learning Objectives

By the end of this chapter, you will be able to:

  • Write basic Markdown — headings, emphasis, lists, links, code blocks, and tables — in notebooks, README files, and GitHub.
  • Structure a notebook with alternating narrative Markdown and code cells using the goal/method/interpretation pattern.
  • Maintain a Known Issues Log and a Data Dictionary as living documents while you work.
  • Write code comments that explain why rather than what.
  • Read a Python traceback from the bottom up and recognize the most common exceptions (FileNotFoundError, KeyError, TypeError, ValueError, AttributeError, ModuleNotFoundError).
  • Apply a systematic five-step debugging process rather than guessing.

2.2 Why This Chapter Matters

Documentation is how you talk to your future self. Error messages are how Python talks back to you. Both are forms of writing, both are forms of diagnostic information, and both are skills that make every other skill in this book easier. When you write clearly about what you are doing and learn to read clearly what the interpreter tells you, “mysterious failures” stop being mysterious.

A persistent fantasy in data work is that you will finish the analysis first and document it later. In practice, “later” never arrives, and by then you have forgotten the reasoning. Why did you drop those 23 rows? Why did you recode "OTH" as missing? Why did you use a left join? Decisions are judgment calls, not mechanical steps; they have to be recorded when you make them, not reconstructed afterward.

If reading a red-text traceback makes your stomach drop, you are in good company. Every programmer feels that at some point. The difference between a beginner and an expert is not that experts stop getting errors — they get plenty. They have built the habit of reading what the error actually says before reacting to the feeling of having broken something.

2.3 Markdown Basics

Markdown is a lightweight formatting language used in Jupyter notebooks, README files, GitHub, Quarto, and many other tools. Plain text with a few markers becomes formatted output when rendered.

2.3.1 Headings

Use # symbols for headings. More # symbols mean smaller headings:

# Top-Level Heading
## Section Heading
### Subsection Heading

2.3.2 Emphasis

*italic text*
**bold text**
`inline code`

2.3.3 Lists

Unordered lists use dashes or asterisks; ordered lists use numbers:

- First item
- Second item

1. Step one
2. Step two

2.3.5 Code blocks

For multi-line code, use triple backticks with an optional language identifier:

```python
df = pd.read_csv('data.csv')
df.head()
```

2.3.6 Tables

| Column | Type | Description |
|--------|------|-------------|
| county | str  | County name |
| year   | int  | Calendar year |

These few elements cover almost everything you need for data documentation. You do not need to learn the full Markdown specification — these basics are enough.

2.4 Documentation in Jupyter Notebooks

A Jupyter notebook interleaves Markdown cells with code cells, which makes it the ideal environment for literate analysis — explanations sitting directly alongside the code they describe.

2.4.1 The goal / method / interpretation pattern

Before each code section, add a Markdown cell explaining what you are about to do and why. After a code section that produces output, add a cell that interprets the result. Think of three elements: the goal (what question are you answering?), the method (what operation are you performing?), and the interpretation (what did you learn?).

A notebook with interleaved narrative and code tells a story. A notebook with only code is a puzzle your future self has to solve.

2.4.2 The Known Issues Log

One of the most valuable documentation habits is a running log of problems and decisions. Place a Markdown cell near the top or bottom of your notebook and add entries as you work:

## Known Issues and Cleaning Decisions

### 2026-02-17: Party field standardization
**Problem:** Party stored inconsistently — "Dem", "DEM", "D", "Democrat"
all mean the same thing.
**Fix:** Created party_map dictionary, normalized to "DEM", "REP", "UNA".
**Impact:** Reduced unique values from 47 to 3; no rows lost.
**Remaining issue:** 23 rows have party="OTH" — unclear what this means.

Each entry records four things: problem, fix, impact (especially any data loss), and any remaining uncertainty. Writing the entry before you run the cleaning cell — as a plan, not a post-hoc summary — keeps the log synchronized with the code.

2.4.3 The Data Dictionary

A Data Dictionary describes each column in your dataset. It is the single most useful piece of documentation you can write, and it takes only a few minutes:

| Column | Type | Description | Valid Values | Cleaning Applied |
|--------|------|-------------|--------------|------------------|
| county | str  | County name | 64 CO counties | .str.strip().str.upper() |
| year   | int  | Calendar year | 2014–2024 | pd.to_numeric() |
| sales  | float | Sales in USD | >= 0 | Removed $ and commas |

Update it whenever you add derived columns or recode values. It is a living document, not a one-time artifact.

2.5 Comments in Code

Markdown cells handle narrative documentation. Code comments handle the technical layer — explaining why a line of code does what it does, not what (which should be clear from the code itself).

# Good comment: explains WHY
# Replace 0 AB with NaN to avoid division-by-zero in batting average
batting['BA'] = batting['H'] / batting['AB'].replace(0, float('nan'))

# Bad comment: restates WHAT the code does
# Divide H by AB
batting['BA'] = batting['H'] / batting['AB']

Useful heuristic: if the code does exactly what it says (df.head(), print(x)), it does not need a comment. If it involves a non-obvious decision — why errors='coerce', why a left join, why AB >= 100 — that is worth a comment.

2.6 Reading Error Messages

Error messages are not punishments. They are diagnostic information — often the most useful information you’ll receive when something goes wrong. Learning to read them systematically is one of the highest-leverage skills you can develop.

When Python encounters a problem, it produces a traceback — a report showing the sequence of function calls that led to the error, ending with the specific error type and message:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
Cell In[3], line 1
----> 1 df = pd.read_csv('my_data.csv')

File ~/anaconda3/lib/python3.11/site-packages/pandas/io/parsers/readers.py:948
    948     return _read(filepath_or_buffer, kwds)

FileNotFoundError: [Errno 2] No such file or directory: 'my_data.csv'

Read from the bottom up. The last line tells you what went wrong. The first ----> line tells you which line of your code triggered it. The middle lines are pandas’ internal code — you can usually skip them.

The two most important pieces of information are always at the bottom: the error type (e.g., FileNotFoundError) and the error message (e.g., No such file or directory: 'my_data.csv').

2.7 Common Error Types

2.7.1 FileNotFoundError

What it means: Python looked for the file at the path you gave and could not find it there relative to where your code is running.

How to fix it: Check the spelling and capitalization. Verify the working directory with import os; print(os.getcwd()). If the file is in a subdirectory, use a relative path like data/raw/my_data.csv.

2.7.2 KeyError

What it means: You tried to access a column (or dict key) that does not exist. Almost always a spelling or capitalization problem.

How to fix it: Print df.columns.tolist(). Watch for trailing spaces ('County ' vs 'County'), inconsistent capitalization, and unexpected characters.

2.7.3 TypeError

What it means: You tried to perform an operation on a type that does not support it — for instance, adding a number to a string. In pandas, this often happens when a column you expected to be numeric is stored as text because it contains non-numeric values like $1,234 or N/A.

How to fix it: Check the dtype with df['column'].dtype. If it shows object when you expected a number, inspect df['column'].unique() to find the non-numeric entries, then clean them before converting.

2.7.4 ValueError

What it means: A conversion received a value it cannot handle — commonly, trying to cast a column containing NaN to integer type.

How to fix it: Either fill or drop NaNs before converting, or use pandas’ nullable integer type (Int64 with a capital I).

2.7.5 AttributeError

What it means: You tried to use a method that does not exist on that object. For example, .str only works on a Series, not a whole DataFrame.

How to fix it: Confirm you are calling the method on the right object. df['column'].str.strip(), not df.str.strip().

2.7.6 ModuleNotFoundError

What it means: Python cannot find the package you tried to import. Either it is not installed, or your notebook is running in a different environment than the one where you installed it.

How to fix it: conda install pandas or pip install pandas. If it is already installed, confirm that your notebook’s kernel matches the environment (see Chapter 1).

2.8 The DtypeWarning

Not an error, but common enough to deserve attention:

DtypeWarning: Columns (3) have mixed types. Specify dtype option on import
or set low_memory=False.

What it means: pandas found both numeric and non-numeric values in the same column and defaulted to storing everything as strings (object type). Arithmetic on that column will not work as expected.

How to fix it: Investigate the column, then clean the non-numeric values or specify the type explicitly at load time with dtype={'col': str}.

2.9 A Systematic Debugging Process

When you encounter an error, resist the urge to randomly change things and re-run. Instead:

  1. Read the last line of the traceback. What is the error type? What does the message say?
  2. Read the first arrow line. Which line of your code caused the error?
  3. Reproduce minimally. Can you trigger the same error with a simpler version of the operation? Break chained method calls into individual steps.
  4. Inspect the inputs. Print the shape, dtype, and a sample of the data going into the operation that failed. Is it what you expected?
  5. Search with the error message. Copy the error type and message into a search engine. Someone has almost certainly hit the same problem.
TipA mindset shift

Beginners often feel that getting an error means they did something wrong. Experienced programmers get errors constantly — the difference is that they read the error message, understand what it says, and use it to fix the problem. Errors are information, not failures.

2.10 Common Issues in This Chapter

WarningYour Markdown renders as raw text instead of formatted output

What it means: The cell in Jupyter is set to “Code” instead of “Markdown.”

How to fix it: Select the cell and press M (in command mode, not edit mode), or use the dropdown at the top of the notebook. Rerun the cell with Shift+Enter.

WarningYour Known Issues Log gets out of sync with the code

What it means: You cleaned something and forgot to log it. Weeks later, you cannot tell why a row count changed.

How to fix it: Write the log entry before running the cleaning cell, as a plan. The log entry is the intent; the code is the execution.

WarningYour notebook is all code with no narrative

What it means: You have a working analysis but no one reading it — including you in three months — can tell what question you were answering or what you concluded.

How to fix it: Add a top-level Markdown cell with the research question. Before each section of code, add a one-sentence goal. After each result, add a one-sentence interpretation.

WarningYou tried every error-fix you could think of and nothing worked

What it means: You’ve stopped reading the error and started guessing. Guessing is slower than reading.

How to fix it: Go back to step 1. Reread the last line of the traceback carefully. Print the inputs to the failing call. If you are still stuck after 15 minutes, search the exact error message verbatim.

2.11 Key Takeaways

  • Documentation is a habit, not a final step. The Known Issues Log and Data Dictionary are living documents you maintain while working.
  • Markdown is tiny — headings, emphasis, lists, links, code blocks, tables — and covers almost everything you need for data documentation.
  • Code comments should explain why you made a decision, not what a line of code does.
  • Read tracebacks from the bottom up. The last line tells you the error type and message. The first arrow line tells you where in your code it happened.
  • Reproduce minimally. Break chained expressions into single steps. Print the inputs. Search the exact error message.
  • Errors are feedback, not failures. The same message that feels like a wall today will feel like a map once you have read a few dozen.

2.12 Further Reading

2.13 Exercises

Open one of the course datasets and identify a data quality issue (a column with mixed types, an unexpected value, missing data in a key field). Write a Known Issues Log entry using the four-part structure: Problem, Fix, Impact, Remaining Issue.

Choose a dataset you have worked with recently. Write a data dictionary as a Markdown table that describes every column, its type, valid values, and any cleaning that was applied. If you have not cleaned the data yet, note what cleaning you think will be needed.

Given the traceback below, identify: (a) the error type, (b) the specific problem described in the message, (c) which line of user code caused the error, and (d) what you would do to fix it.

---------------------------------------------------------------------------
KeyError                                  Traceback (most recent call last)
Cell In[7], line 1
----> 1 df['Retial_Sales'].sum()

KeyError: 'Retial_Sales'
  1. KeyError. (b) The column 'Retial_Sales' does not exist — note the typo: “Retial” instead of “Retail”. (c) Line 1 of cell 7. (d) Correct the column name to 'Retail_Sales' — confirm the exact name with df.columns.tolist().