2 Documentation, Markdown, and Errors
2.1 Learning Objectives
By the end of this chapter, you will be able to:
- Write basic Markdown — headings, emphasis, lists, links, code blocks, and tables — in notebooks, README files, and GitHub.
- Structure a notebook with alternating narrative Markdown and code cells using the goal/method/interpretation pattern.
- Maintain a Known Issues Log and a Data Dictionary as living documents while you work.
- Write code comments that explain why rather than what.
- Read a Python traceback from the bottom up and recognize the most common exceptions (
FileNotFoundError,KeyError,TypeError,ValueError,AttributeError,ModuleNotFoundError). - Apply a systematic five-step debugging process rather than guessing.
2.2 Why This Chapter Matters
Documentation is how you talk to your future self. Error messages are how Python talks back to you. Both are forms of writing, both are forms of diagnostic information, and both are skills that make every other skill in this book easier. When you write clearly about what you are doing and learn to read clearly what the interpreter tells you, “mysterious failures” stop being mysterious.
A persistent fantasy in data work is that you will finish the analysis first and document it later. In practice, “later” never arrives, and by then you have forgotten the reasoning. Why did you drop those 23 rows? Why did you recode "OTH" as missing? Why did you use a left join? Decisions are judgment calls, not mechanical steps; they have to be recorded when you make them, not reconstructed afterward.
If reading a red-text traceback makes your stomach drop, you are in good company. Every programmer feels that at some point. The difference between a beginner and an expert is not that experts stop getting errors — they get plenty. They have built the habit of reading what the error actually says before reacting to the feeling of having broken something.
2.3 Markdown Basics
Markdown is a lightweight formatting language used in Jupyter notebooks, README files, GitHub, Quarto, and many other tools. Plain text with a few markers becomes formatted output when rendered.
2.3.1 Headings
Use # symbols for headings. More # symbols mean smaller headings:
# Top-Level Heading
## Section Heading
### Subsection Heading2.3.2 Emphasis
*italic text*
**bold text**
`inline code`2.3.3 Lists
Unordered lists use dashes or asterisks; ordered lists use numbers:
- First item
- Second item
1. Step one
2. Step two2.3.4 Links
[Link text](https://example.com)2.3.5 Code blocks
For multi-line code, use triple backticks with an optional language identifier:
```python
df = pd.read_csv('data.csv')
df.head()
```2.3.6 Tables
| Column | Type | Description |
|--------|------|-------------|
| county | str | County name |
| year | int | Calendar year |These few elements cover almost everything you need for data documentation. You do not need to learn the full Markdown specification — these basics are enough.
2.4 Documentation in Jupyter Notebooks
A Jupyter notebook interleaves Markdown cells with code cells, which makes it the ideal environment for literate analysis — explanations sitting directly alongside the code they describe.
2.4.1 The goal / method / interpretation pattern
Before each code section, add a Markdown cell explaining what you are about to do and why. After a code section that produces output, add a cell that interprets the result. Think of three elements: the goal (what question are you answering?), the method (what operation are you performing?), and the interpretation (what did you learn?).
A notebook with interleaved narrative and code tells a story. A notebook with only code is a puzzle your future self has to solve.
2.4.2 The Known Issues Log
One of the most valuable documentation habits is a running log of problems and decisions. Place a Markdown cell near the top or bottom of your notebook and add entries as you work:
## Known Issues and Cleaning Decisions
### 2026-02-17: Party field standardization
**Problem:** Party stored inconsistently — "Dem", "DEM", "D", "Democrat"
all mean the same thing.
**Fix:** Created party_map dictionary, normalized to "DEM", "REP", "UNA".
**Impact:** Reduced unique values from 47 to 3; no rows lost.
**Remaining issue:** 23 rows have party="OTH" — unclear what this means.Each entry records four things: problem, fix, impact (especially any data loss), and any remaining uncertainty. Writing the entry before you run the cleaning cell — as a plan, not a post-hoc summary — keeps the log synchronized with the code.
2.4.3 The Data Dictionary
A Data Dictionary describes each column in your dataset. It is the single most useful piece of documentation you can write, and it takes only a few minutes:
| Column | Type | Description | Valid Values | Cleaning Applied |
|--------|------|-------------|--------------|------------------|
| county | str | County name | 64 CO counties | .str.strip().str.upper() |
| year | int | Calendar year | 2014–2024 | pd.to_numeric() |
| sales | float | Sales in USD | >= 0 | Removed $ and commas |Update it whenever you add derived columns or recode values. It is a living document, not a one-time artifact.
2.6 Reading Error Messages
Error messages are not punishments. They are diagnostic information — often the most useful information you’ll receive when something goes wrong. Learning to read them systematically is one of the highest-leverage skills you can develop.
When Python encounters a problem, it produces a traceback — a report showing the sequence of function calls that led to the error, ending with the specific error type and message:
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
Cell In[3], line 1
----> 1 df = pd.read_csv('my_data.csv')
File ~/anaconda3/lib/python3.11/site-packages/pandas/io/parsers/readers.py:948
948 return _read(filepath_or_buffer, kwds)
FileNotFoundError: [Errno 2] No such file or directory: 'my_data.csv'
Read from the bottom up. The last line tells you what went wrong. The first ----> line tells you which line of your code triggered it. The middle lines are pandas’ internal code — you can usually skip them.
The two most important pieces of information are always at the bottom: the error type (e.g., FileNotFoundError) and the error message (e.g., No such file or directory: 'my_data.csv').
2.7 Common Error Types
2.7.1 FileNotFoundError
What it means: Python looked for the file at the path you gave and could not find it there relative to where your code is running.
How to fix it: Check the spelling and capitalization. Verify the working directory with import os; print(os.getcwd()). If the file is in a subdirectory, use a relative path like data/raw/my_data.csv.
2.7.2 KeyError
What it means: You tried to access a column (or dict key) that does not exist. Almost always a spelling or capitalization problem.
How to fix it: Print df.columns.tolist(). Watch for trailing spaces ('County ' vs 'County'), inconsistent capitalization, and unexpected characters.
2.7.3 TypeError
What it means: You tried to perform an operation on a type that does not support it — for instance, adding a number to a string. In pandas, this often happens when a column you expected to be numeric is stored as text because it contains non-numeric values like $1,234 or N/A.
How to fix it: Check the dtype with df['column'].dtype. If it shows object when you expected a number, inspect df['column'].unique() to find the non-numeric entries, then clean them before converting.
2.7.4 ValueError
What it means: A conversion received a value it cannot handle — commonly, trying to cast a column containing NaN to integer type.
How to fix it: Either fill or drop NaNs before converting, or use pandas’ nullable integer type (Int64 with a capital I).
2.7.5 AttributeError
What it means: You tried to use a method that does not exist on that object. For example, .str only works on a Series, not a whole DataFrame.
How to fix it: Confirm you are calling the method on the right object. df['column'].str.strip(), not df.str.strip().
2.7.6 ModuleNotFoundError
What it means: Python cannot find the package you tried to import. Either it is not installed, or your notebook is running in a different environment than the one where you installed it.
How to fix it: conda install pandas or pip install pandas. If it is already installed, confirm that your notebook’s kernel matches the environment (see Chapter 1).
2.8 The DtypeWarning
Not an error, but common enough to deserve attention:
DtypeWarning: Columns (3) have mixed types. Specify dtype option on import
or set low_memory=False.
What it means: pandas found both numeric and non-numeric values in the same column and defaulted to storing everything as strings (object type). Arithmetic on that column will not work as expected.
How to fix it: Investigate the column, then clean the non-numeric values or specify the type explicitly at load time with dtype={'col': str}.
2.9 A Systematic Debugging Process
When you encounter an error, resist the urge to randomly change things and re-run. Instead:
- Read the last line of the traceback. What is the error type? What does the message say?
- Read the first arrow line. Which line of your code caused the error?
- Reproduce minimally. Can you trigger the same error with a simpler version of the operation? Break chained method calls into individual steps.
- Inspect the inputs. Print the shape, dtype, and a sample of the data going into the operation that failed. Is it what you expected?
- Search with the error message. Copy the error type and message into a search engine. Someone has almost certainly hit the same problem.
Beginners often feel that getting an error means they did something wrong. Experienced programmers get errors constantly — the difference is that they read the error message, understand what it says, and use it to fix the problem. Errors are information, not failures.
2.10 Common Issues in This Chapter
What it means: The cell in Jupyter is set to “Code” instead of “Markdown.”
How to fix it: Select the cell and press M (in command mode, not edit mode), or use the dropdown at the top of the notebook. Rerun the cell with Shift+Enter.
What it means: You cleaned something and forgot to log it. Weeks later, you cannot tell why a row count changed.
How to fix it: Write the log entry before running the cleaning cell, as a plan. The log entry is the intent; the code is the execution.
What it means: You have a working analysis but no one reading it — including you in three months — can tell what question you were answering or what you concluded.
How to fix it: Add a top-level Markdown cell with the research question. Before each section of code, add a one-sentence goal. After each result, add a one-sentence interpretation.
What it means: You’ve stopped reading the error and started guessing. Guessing is slower than reading.
How to fix it: Go back to step 1. Reread the last line of the traceback carefully. Print the inputs to the failing call. If you are still stuck after 15 minutes, search the exact error message verbatim.
2.11 Key Takeaways
- Documentation is a habit, not a final step. The Known Issues Log and Data Dictionary are living documents you maintain while working.
- Markdown is tiny — headings, emphasis, lists, links, code blocks, tables — and covers almost everything you need for data documentation.
- Code comments should explain why you made a decision, not what a line of code does.
- Read tracebacks from the bottom up. The last line tells you the error type and message. The first arrow line tells you where in your code it happened.
- Reproduce minimally. Break chained expressions into single steps. Print the inputs. Search the exact error message.
- Errors are feedback, not failures. The same message that feels like a wall today will feel like a map once you have read a few dozen.
2.12 Further Reading
- Markdown Guide — markdownguide.org is the cleanest reference; the “Cheat Sheet” page is worth bookmarking.
- GitHub Flavored Markdown Spec — github.github.com/gfm/ covers GitHub’s dialect, including task lists and tables.
- Python Built-in Exceptions — docs.python.org/3/library/exceptions.html lists every exception type in the standard library.
- Python Errors and Exceptions tutorial — docs.python.org/3/tutorial/errors.html is the official tutorial including
try/except. - Real Python: Python Exceptions — realpython.com/python-exceptions/ is an accessible walkthrough with examples.
- pandas: Common pitfalls — pandas.pydata.org/docs/user_guide/gotchas.html covers pandas-specific behavior that commonly surprises newcomers.
2.13 Exercises
Open one of the course datasets and identify a data quality issue (a column with mixed types, an unexpected value, missing data in a key field). Write a Known Issues Log entry using the four-part structure: Problem, Fix, Impact, Remaining Issue.
Choose a dataset you have worked with recently. Write a data dictionary as a Markdown table that describes every column, its type, valid values, and any cleaning that was applied. If you have not cleaned the data yet, note what cleaning you think will be needed.
Given the traceback below, identify: (a) the error type, (b) the specific problem described in the message, (c) which line of user code caused the error, and (d) what you would do to fix it.
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
Cell In[7], line 1
----> 1 df['Retial_Sales'].sum()
KeyError: 'Retial_Sales'
KeyError. (b) The column'Retial_Sales'does not exist — note the typo: “Retial” instead of “Retail”. (c) Line 1 of cell 7. (d) Correct the column name to'Retail_Sales'— confirm the exact name withdf.columns.tolist().
2.5 Comments in Code
Markdown cells handle narrative documentation. Code comments handle the technical layer — explaining why a line of code does what it does, not what (which should be clear from the code itself).
Useful heuristic: if the code does exactly what it says (
df.head(),print(x)), it does not need a comment. If it involves a non-obvious decision — whyerrors='coerce', why a left join, whyAB >= 100— that is worth a comment.