Code Quality & Environment Tools

Introduction

Most of this course has been about doing things with data: loading it, modeling it, visualizing it, serving it. This lesson is different. It is about the tools that keep the code around all of that clean, consistent, and safe — the kind of dependencies you install once per project and then mostly forget about, until the day they save you from a messy diff or a leaked API key.

These three packages are not data science libraries in the traditional sense. They do not touch your DataFrames or your models. But almost every serious Python data science repository installs them, so understanding what problem each one solves is part of knowing the ecosystem.

black: The Uncompromising Formatter

black is an opinionated code formatter. "Opinionated" means it does not offer dozens of configuration options for how your code should look — it has one style, and it applies that style to your entire codebase automatically. The pitch is simple: stop spending time and energy debating tabs versus spaces or where to put a trailing comma, and let a tool decide for you.

pip install black

Run it against a single file, a folder, or your whole project. It rewrites files in place.

# Format the current project in place
black .

# Check what would change without writing anything (useful in CI)
black --check .

def load_data(path,   columns=None):
    df = pd.read_csv(path,index_col = 0)
    if columns:
        df=df[columns]
    return df

def load_data(path, columns=None):
    df = pd.read_csv(path, index_col=0)
    if columns:
        df = df[columns]
    return df

ruff: An Extremely Fast Linter

A linter checks your code for problems that are not syntax errors: unused imports, undefined names, variables that are never used, lines that are too long, common bug patterns. ruff does this job and has largely replaced older tools like flake8, pylint, and isort in new projects, mainly because it is written in Rust and runs dramatically faster — often 10 to 100 times faster than the tools it replaces, which matters a lot once a project has thousands of files.

pip install ruff

# Lint the project and report problems
ruff check .

# Automatically fix the problems ruff knows how to fix
ruff check . --fix

# ruff can also format code, similar to black
ruff format .

import pandas as pd
import numpy as np  # unused import

def summarize(df):
    total = df['sales'].sum()
    reuslt = total * 1.1  # typo, and never used
    return total

python-dotenv: Keeping Secrets Out of Your Code

Data science projects almost always need some kind of secret or environment-specific config: a database password, an API key for a paid data source, a file path that differs between your laptop and a server. Hardcoding these values directly into a Python script is a common mistake — it means secrets end up committed to version control, and config has to be edited by hand every time it changes.

python-dotenv solves this by loading key-value pairs from a plain text .env file into environment variables, which your code then reads normally with os.environ. The .env file itself is added to .gitignore, so it never gets committed.

pip install python-dotenv

DATABASE_URL=postgresql://user:password@localhost:5432/salesdb
API_KEY=sk_live_example_key_123

import os
from dotenv import load_dotenv

load_dotenv()  # reads .env and populates os.environ

database_url = os.environ["DATABASE_URL"]
api_key = os.environ.get("API_KEY")

print("Connecting with a key that is never hardcoded in this file.")

What Each Tool Solves

Common Mistakes

Best Practices

Frequently Asked Questions

Key Takeaways

Summary

black, ruff, and python-dotenv are not data science libraries, but they are dependencies you will find in nearly every serious Python data project. They keep code formatted consistently, catch mistakes before they become bugs, and keep secrets out of your git history. Next, we move back into the data layer, looking at how Python talks to databases and REST APIs.