LearnAI ToolsCareerPractice BuildsPlayContact
Lesson 319 min read

pip vs conda: Managing Data Science Dependencies

Compare pip and conda, understand why conda exists for binary dependencies, and learn the difference between requirements.txt and environment.yml.

Introduction

Every data science project eventually asks the same question: pip or conda? Both install Python packages, but they solve slightly different problems, and picking the wrong one for a given project can mean hours lost to broken installs.

This lesson compares the two directly, explains why conda exists at all when pip already works, and covers the dependency files — requirements.txt and environment.yml — that each one relies on.

What You Will Learn
  • What pip and conda each do, with example commands.
  • Why conda exists — specifically for non-Python binary dependencies like CUDA and BLAS.
  • The difference between requirements.txt and environment.yml.
  • A practical comparison table and guidance on when teams pick each tool.

What is pip?

pip is Python's default package installer. It installs pure-Python packages (and many packages with precompiled "wheel" binaries) directly from PyPI.

pip install numpy pandas scikit-learn

pip is lightweight, ships with Python itself, and is the standard choice for most day-to-day Python development, web projects, and simple data science work.

What is conda?

conda is both a package manager and an environment manager, developed by Anaconda. It manages Python packages too, but it is not limited to Python — it can install entire binary toolchains, including specific compiler versions, R packages, and GPU libraries.

conda install numpy pandas scikit-learn

Why Conda Exists: Binary Dependencies

Many data science and ML packages are not pure Python. Under the hood, NumPy relies on BLAS/LAPACK linear algebra libraries, and PyTorch or TensorFlow rely on CUDA and cuDNN for GPU acceleration. These are compiled, non-Python libraries that pip was never designed to manage directly.

A Concrete Example

Installing a GPU-enabled version of PyTorch with pip requires matching your exact CUDA driver version by hand. conda instead ships the correct CUDA toolkit alongside the package, resolving the whole binary stack in one command.

conda install pytorch torchvision cudatoolkit=11.8 -c pytorch

This is the core reason conda persists in data science specifically: it treats non-Python binary dependencies as first-class citizens instead of leaving you to install system libraries manually.

requirements.txt vs environment.yml

Both tools use a manifest file to describe a project's dependencies so they can be recreated elsewhere. pip uses a flat requirements.txt; conda uses a richer environment.yml.

requirements.txt
numpy==1.26.4
pandas==2.2.2
scikit-learn==1.4.2
environment.yml
name: ds-project
channels:
  - conda-forge
dependencies:
  - python=3.11
  - numpy=1.26.4
  - pandas=2.2.2
  - pip
  - pip:
      - some-pip-only-package==0.3.1

Notice that environment.yml can pin the Python version itself and even fall back to pip for packages that are not available through conda channels — the two tools are often used together, not strictly as alternatives.

pip vs conda at a Glance

Aspectpipconda
SourcePyPIAnaconda repositories / conda-forge
ScopePython packages onlyPython packages plus system-level binaries
Handles GPU/CUDA wellManually, error-proneYes, built in
Environment managementNeeds venv separatelyBuilt in (conda create/activate)
Manifest filerequirements.txtenvironment.yml
Install speedGenerally fastCan be slower to resolve dependencies
Ships with PythonYesNo, requires Anaconda/Miniconda install

When Teams Pick One Over the Other

  • Web backends and general-purpose Python projects almost always use pip — there is no binary dependency problem to solve.
  • Deep learning teams working with GPUs often default to conda, or to pip with pre-built wheels that already bundle CUDA (like recent PyTorch releases).
  • Academic and scientific computing groups frequently standardize on conda because it also manages non-Python tools common in research (R, Jupyter kernels, compilers).
  • Production deployment pipelines often prefer pip with a locked requirements.txt because it produces smaller, more predictable container images.

Common Mistakes

Avoid These Mistakes
  • Freely mixing pip install and conda install in the same environment — this can silently corrupt the environment's dependency graph.
  • Forgetting to pin versions in requirements.txt, causing "works on my machine" bugs when teammates install different versions later.
  • Assuming conda is always slower or always better — the right choice depends on whether binary dependencies are actually involved.

Best Practices

  • If your project touches GPUs, CUDA, or heavy compiled scientific libraries, default to conda or vendor-provided wheels.
  • If your project is pure Python or web-facing, default to pip for its speed and simplicity.
  • If you must mix them, install conda packages first, then use pip only for what conda does not provide — never the reverse.

Frequently Asked Questions

Yes, and it is common — but install conda packages first, then use pip for anything conda-forge does not have, to minimize conflicts.

No, but data science and ML are where its binary-dependency handling shines brightest, which is why it is so strongly associated with the field.

No — Miniconda installs just the conda package manager without Anaconda's large bundle of pre-installed packages, and is the lighter-weight choice for most people.

Key Takeaways

  • pip installs Python packages from PyPI; conda manages both Python packages and non-Python binary dependencies.
  • conda exists primarily to solve the binary dependency problem — things like CUDA, BLAS, and compiled scientific libraries.
  • requirements.txt (pip) and environment.yml (conda) both describe a project's dependencies for reproducibility.
  • Teams choose based on whether their stack has heavy binary/GPU dependencies or is mostly pure Python.

Summary

pip and conda are not strict competitors — they solve overlapping but distinct problems. pip is the lightweight default; conda earns its place when binary dependencies like CUDA enter the picture.

Lesson 3 Completed
  • You understand what pip and conda each do.
  • You know why conda exists for binary/GPU dependencies.
  • You can choose the right manifest file for a project.
Next Lesson →

Virtual Environments & Requirements Files