16 Python Environment Setup

This appendix provides instructions for setting up the Python environment needed to run the code in this book. All code was developed and tested with the Anaconda distribution of Python 3.10 running in Jupyter Notebook.

16.1 Installing Anaconda

We recommend the Anaconda distribution of Python, which bundles Python with the most commonly used data science libraries. Download the installer for your operating system from https://www.anaconda.com/download and follow the installation instructions. Choose Python 3.10 or above.

Anaconda includes all of the core libraries used in this book: numpy, pandas, matplotlib, seaborn, scipy, scikit-learn, and statsmodels. It also includes Jupyter Notebook and JupyterLab, the interactive computing environments in which this book’s code is designed to run.

16.2 Verifying Your Installation

After installing Anaconda, open a terminal (macOS/Linux) or Anaconda Prompt (Windows) and verify your Python version and key library installations.

python --version

You should see Python 3.10 or higher. Then verify the key libraries.

python -c "import pandas; print('pandas', pandas.__version__)"
python -c "import matplotlib; print('matplotlib', matplotlib.__version__)"
python -c "import seaborn; print('seaborn', seaborn.__version__)"
python -c "import scipy; print('scipy', scipy.__version__)"
python -c "import sklearn; print('scikit-learn', sklearn.__version__)"
python -c "import statsmodels; print('statsmodels', statsmodels.__version__)"

16.3 Launching Jupyter

To start a Jupyter Notebook server, navigate to the directory containing the book’s data files and run one of the following commands.

jupyter notebook

jupyter lab

This will open a browser window where you can create new notebooks or open existing .ipynb files. The classic Notebook interface (jupyter notebook) is simpler and more widely documented. The JupyterLab interface (jupyter lab) offers a more modern multi-pane layout with a file browser, terminal, and multiple notebook tabs.

16.4 Required Libraries

The following libraries are used throughout the book. All are included with the Anaconda distribution.

Library	Purpose	Import Convention	First Used
`numpy`	Numerical computing and arrays	`import numpy as np`	Chapter 1
`pandas`	Data manipulation and analysis	`import pandas as pd`	Chapter 1
`matplotlib`	Core data visualization	`import matplotlib.pyplot as plt`	Chapter 5
`seaborn`	Statistical visualization	`import seaborn as sb`	Chapter 7
`scipy`	Statistical functions and tests	`from scipy import stats`	Chapter 7
`scikit-learn`	Machine learning models	`from sklearn.linear_model import LinearRegression`	Chapter 13
`statsmodels`	Statistical modeling and diagnostics	`import statsmodels.formula.api as smf`	Chapter 13
`datetime`	Date and time handling	`import datetime`	Chapter 9
`os`	File system operations	`import os`	Chapter 3

Import conventions matter

The aliases shown above (np, pd, plt, sb) are widely used conventions in the Python data science community. Using them makes your code consistent with tutorials, documentation, and Stack Overflow answers. Note that some resources import seaborn as sns rather than sb — both work, and the choice is a matter of preference.

16.4.1 Optional Libraries

Two additional libraries appear in specific chapters but are not required for the majority of the book.

wikifunctions is a custom library for retrieving data from Wikipedia APIs, used in Chapter 10. It is not available via conda or pip and must be downloaded from https://github.com/brianckeegan/wikifunctions/ and placed in the same directory as the notebook.

ipympl enables interactive matplotlib widgets in Jupyter Notebook and is used in Chapter 13 for interactive 3D regression visualization. Install it with conda install ipympl -c conda-forge if you want to use the %matplotlib widget magic command.

16.5 Installing Additional Packages

If any library is missing, install it using conda (preferred for compatibility) or pip.

conda install seaborn

pip install seaborn

For pip installations in environments managed by Anaconda, you may need the --break-system-packages flag on newer Python versions.

16.6 Common Setup Block

Most chapters begin with a standard setup block. You can copy this into the first cell of any notebook.

import numpy as np
import pandas as pd

pd.options.display.max_columns = 100

%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sb

The %matplotlib inline cell magic ensures that plots render directly in the notebook rather than opening in a separate window. The pd.options.display.max_columns setting prevents pandas from truncating wide DataFrames when you call .head() or display a DataFrame.

16.7 Getting the Data

Clone the book’s repository to download all datasets.

git clone https://github.com/cuinfoscience/Information-Exposition-Book.git
cd Information-Exposition-Book

See Chapter 15 for a description of each dataset file, including its source, format, and any special loading parameters required (such as sep='|' for pipe-delimited files or na_values='..' for World Bank data).

16.8 Working Directory

Many code cells in this book load data using relative file paths like pd.read_csv('sp500.csv'). This assumes that the notebook’s working directory is the same directory that contains the data files. You can check and change the working directory with the following code.

import os
print(os.getcwd())

# Uncomment and modify to change directory
# os.chdir('/path/to/data/directory')

If you organize your data into subdirectories (e.g., data/Retrosheet/, data/Lahman/), you will need to adjust the file paths in the code accordingly. The os.listdir() function, introduced in Chapter 3, can help you explore directory contents.

16.9 Troubleshooting

ModuleNotFoundError when importing a library. The library is not installed in your current Python environment. Use conda install or pip install to add it. If you have multiple Python environments, make sure you are installing into the correct one.

Plots do not appear in the notebook. Ensure that %matplotlib inline appears in a cell that has been executed before any plotting code. If you restart the kernel, you need to re-execute the setup cell.

FileNotFoundError when calling read_csv. The notebook’s working directory does not match the location of the data files. Use os.getcwd() to check where the notebook thinks it is, and either move the data or change the working directory.

Pandas displays ... instead of all columns. The DataFrame has more columns than the display limit. Increase the limit with pd.options.display.max_columns = 200 or use .head().T to transpose the display.

SettingWithCopyWarning. This warning indicates that you are modifying a “view” of a DataFrame rather than an independent copy. To silence it and ensure correct behavior, use .copy() when creating a filtered DataFrame, as described in Chapter 12.

Memory errors with large datasets. If you are working with the full Retrosheet game log concatenation (150,000+ rows × 161 columns) on a machine with limited RAM, consider using the nrows parameter in read_csv to work with a subset, or use dtype to specify memory-efficient types for columns you don’t need at full precision.

# Python Environment Setup {#sec-setup} This appendix provides instructions for setting up the Python environment needed to run the code in this book. All code was developed and tested with the Anaconda distribution of Python 3.10 running in Jupyter Notebook. ## Installing Anaconda We recommend the Anaconda distribution of Python, which bundles Python with the most commonly used data science libraries. Download the installer for your operating system from <https://www.anaconda.com/download> and follow the installation instructions. Choose Python 3.10 or above. Anaconda includes all of the core libraries used in this book: numpy, pandas, matplotlib, seaborn, scipy, scikit-learn, and statsmodels. It also includes Jupyter Notebook and JupyterLab, the interactive computing environments in which this book's code is designed to run. ## Verifying Your Installation After installing Anaconda, open a terminal (macOS/Linux) or Anaconda Prompt (Windows) and verify your Python version and key library installations. ```bash python --version ``` You should see Python 3.10 or higher. Then verify the key libraries. ```bash python -c "import pandas; print('pandas', pandas.__version__)" python -c "import matplotlib; print('matplotlib', matplotlib.__version__)" python -c "import seaborn; print('seaborn', seaborn.__version__)" python -c "import scipy; print('scipy', scipy.__version__)" python -c "import sklearn; print('scikit-learn', sklearn.__version__)" python -c "import statsmodels; print('statsmodels', statsmodels.__version__)" ``` ## Launching Jupyter To start a Jupyter Notebook server, navigate to the directory containing the book's data files and run one of the following commands. ```bash jupyter notebook ``` ```bash jupyter lab ``` This will open a browser window where you can create new notebooks or open existing `.ipynb` files. The classic Notebook interface (`jupyter notebook`) is simpler and more widely documented. The JupyterLab interface (`jupyter lab`) offers a more modern multi-pane layout with a file browser, terminal, and multiple notebook tabs. ## Required Libraries The following libraries are used throughout the book. All are included with the Anaconda distribution. | Library | Purpose | Import Convention | First Used | |---------|---------|-------------------|------------| | `numpy` | Numerical computing and arrays | `import numpy as np` | @sec-loading | | `pandas` | Data manipulation and analysis | `import pandas as pd` | @sec-loading | | `matplotlib` | Core data visualization | `import matplotlib.pyplot as plt` | @sec-clarity | | `seaborn` | Statistical visualization | `import seaborn as sb` | @sec-catplots | | `scipy` | Statistical functions and tests | `from scipy import stats` | @sec-catplots | | `scikit-learn` | Machine learning models | `from sklearn.linear_model import LinearRegression` | @sec-linear-models | | `statsmodels` | Statistical modeling and diagnostics | `import statsmodels.formula.api as smf` | @sec-linear-models | | `datetime` | Date and time handling | `import datetime` | @sec-temporal-data | | `os` | File system operations | `import os` | @sec-combining | ::: {.callout-tip} ## Import conventions matter The aliases shown above (`np`, `pd`, `plt`, `sb`) are widely used conventions in the Python data science community. Using them makes your code consistent with tutorials, documentation, and Stack Overflow answers. Note that some resources import seaborn as `sns` rather than `sb` — both work, and the choice is a matter of preference. ::: ### Optional Libraries Two additional libraries appear in specific chapters but are not required for the majority of the book. `wikifunctions` is a custom library for retrieving data from Wikipedia APIs, used in @sec-time-series. It is not available via `conda` or `pip` and must be downloaded from <https://github.com/brianckeegan/wikifunctions/> and placed in the same directory as the notebook. `ipympl` enables interactive matplotlib widgets in Jupyter Notebook and is used in @sec-linear-models for interactive 3D regression visualization. Install it with `conda install ipympl -c conda-forge` if you want to use the `%matplotlib widget` magic command. ## Installing Additional Packages If any library is missing, install it using conda (preferred for compatibility) or pip. ```bash conda install seaborn ``` ```bash pip install seaborn ``` For pip installations in environments managed by Anaconda, you may need the `--break-system-packages` flag on newer Python versions. ## Common Setup Block Most chapters begin with a standard setup block. You can copy this into the first cell of any notebook. ```{python} import numpy as np import pandas as pd pd.options.display.max_columns = 100 %matplotlib inline import matplotlib.pyplot as plt import seaborn as sb ``` The `%matplotlib inline` cell magic ensures that plots render directly in the notebook rather than opening in a separate window. The `pd.options.display.max_columns` setting prevents pandas from truncating wide DataFrames when you call `.head()` or display a DataFrame. ## Getting the Data Clone the book's repository to download all datasets. ```bash git clone https://github.com/cuinfoscience/Information-Exposition-Book.git cd Information-Exposition-Book ``` See @sec-datasets for a description of each dataset file, including its source, format, and any special loading parameters required (such as `sep='|'` for pipe-delimited files or `na_values='..'` for World Bank data). ## Working Directory Many code cells in this book load data using relative file paths like `pd.read_csv('sp500.csv')`. This assumes that the notebook's working directory is the same directory that contains the data files. You can check and change the working directory with the following code. ```{python} import os print(os.getcwd()) ``` ```{python} # Uncomment and modify to change directory # os.chdir('/path/to/data/directory') ``` If you organize your data into subdirectories (e.g., `data/Retrosheet/`, `data/Lahman/`), you will need to adjust the file paths in the code accordingly. The `os.listdir()` function, introduced in @sec-combining, can help you explore directory contents. ## Troubleshooting **`ModuleNotFoundError` when importing a library.** The library is not installed in your current Python environment. Use `conda install` or `pip install` to add it. If you have multiple Python environments, make sure you are installing into the correct one. **Plots do not appear in the notebook.** Ensure that `%matplotlib inline` appears in a cell that has been executed *before* any plotting code. If you restart the kernel, you need to re-execute the setup cell. **`FileNotFoundError` when calling `read_csv`.** The notebook's working directory does not match the location of the data files. Use `os.getcwd()` to check where the notebook thinks it is, and either move the data or change the working directory. **Pandas displays `...` instead of all columns.** The DataFrame has more columns than the display limit. Increase the limit with `pd.options.display.max_columns = 200` or use `.head().T` to transpose the display. **`SettingWithCopyWarning`.** This warning indicates that you are modifying a "view" of a DataFrame rather than an independent copy. To silence it and ensure correct behavior, use `.copy()` when creating a filtered DataFrame, as described in @sec-scatterplots. **Memory errors with large datasets.** If you are working with the full Retrosheet game log concatenation (150,000+ rows × 161 columns) on a machine with limited RAM, consider using the `nrows` parameter in `read_csv` to work with a subset, or use `dtype` to specify memory-efficient types for columns you don't need at full precision.