1.3 Python Ecosystem#

What do you need to run your Python code?#

  • Python. Most packages work with Python3. Your laptop may already come with Python preinstalled. We recommend downloading Anaconda, which comes with a version of python, and aliasing \(python\) to the binary executable python specific to conda.

  • Some code: Files with .py extension

  • Dependencies: Specific modules that contain many functions related to each other (e.g. numpy, scipy, pandas, scikit-learn). Often these modules are packages imported in the python code as

    import numpy as np

To run Python code with the proper dependencies, you need to define an environment: Version of Python + Some packages + Version of packages.

What are virtual environments?#

A Python virtual environment is an isolated working copy of a specific version of Python interpreter together with specific versions of a number of external libraries installed into that virtual environment.

  • You may have on your computer different Python codes with different versions of packages

  • You give your code to a friend

  • Some of your packages may depend on other packages, with a specific version. How do you make sure you have the right version of everything?

How to deal with this?#

Install miniconda.

  • User interface + command line

  • Tools for developing code in Python: JupyterLab, Spyder

  • Jupyter notebook (more on this later)

  • Tools for managing environment

Basic conda commands#

  conda info

… to check conda version to make sure its installed.

  conda env list

… to list out available environments (the starred * environment is the current activate environment).

  conda env create --file environment.yml

… to create conda environment from environment file.

  conda env remove --yes --name myenv

… to delete the conda environment.

  conda activate myenv

… to activate a conda enivronment by the env name myenv. Inside the environment, there are other ways to add libraries. Use conda install or pip3 install to add packages that will be automatically saved.

  conda deactivate

… to deactivate the current cona environment.


Example of .yml file#

name: MLlabs
    channels:
      - conda-forge
      - defaults
    dependencies:
      - python=3.9
      - jupyter
      - matplotlib
      - numpy
      - pandas
      - scipy
      - scikit-learn
      - pytorch

To learn more about Anaconda environments, visit this site

How to freeze an environment#

  (venv) $ pip3 freeze > requirements.txt

These will provide the libraries and their very specific versions so that codes can be reproduced in the exact same environment. Example of a requirement file is:

  cycler==0.10.0            # via matplotlib
  kiwisolver==1.2.0         # via matplotlib
  matplotlib==3.2.1
  numpy==1.18.5
  pandas==1.0.4
  pyparsing==2.4.7          # via matplotlib
  python-dateutil==2.8.1    # via matplotlib, pandas
  pytz==2020.1              # via pandas
  scipy==1.4.1              # via seaborn
  seaborn==0.10.1
  six==1.15.0               # via cycler, python-dateutil