nbdev Blog – ES114 C028

Introduction

Python package development can be complex, requiring structured modules, proper documentation, and testing. nbdev, an open-source library from Fast.ai, simplifies this process by allowing developers to create, document, and test Python packages directly within Jupyter Notebooks. This enables a more interactive and efficient development workflow.

Traditional Python package development involves writing scripts, maintaining separate documentation, and setting up unit tests manually. With nbdev, all these steps are integrated into a single environment, reducing friction and increasing productivity.

Installation & Setup

nbdev works on macOS, Linux, and most Unix-style operating systems. It works on Windows under WSL, but not under cmd or Powershell.

For Mac and Linux, enter:

conda install -c fastai -y nbdev

Or for Mac, Linux and Windows: If you’re using pip

pip install nbdev

You can also run nbdev_help from the terminal to see the full list of available commands:

nbdev_help

Note that nbdev must be installed into the same Python environment that you use for both Jupyter and your project.

This initializes a structured project with necessary configurations, including essential files such as:
- settings.ini: Stores project metadata.
- index.ipynb: The main notebook for your package.
- nbs/: A directory containing all Jupyter Notebooks.
- README.md: The auto-generated project description.

Key Features & Explanation

1. Generating Documentation with nbdev

Develop your entire Python package in Jupyter Notebooks, allowing for interactive coding, visualization, and documentation. This eliminates the need for separate .py files.No need to switch between multiple files (code files, test files, and documentation files).

2. Building Documentation in nbdev

With nbdev, documentation is generated directly from docstrings and markdown cells in notebooks. Run the following command to build documentation:

nbdev_build_docs

This extracts markdown and code cells to create a well-structured documentation site.

3. Built-in Testing

nbdev supports unit testing within notebooks. You can write and execute tests using:

nbdev_test

This ensures code correctness without requiring separate test files.

4. Version Control

nbdev provides tools to manage version control and avoid merge conflicts with Jupyter Notebooks. We can use this

nbdev_clean_nbs

This strips unnecessary metadata before committing to Git, making collaboration seamless. ### 5.Type Checking & Linting

nbdev_fix

helps enforce coding style and detect type errors, ensuring clean and professional-quality code.

6. Integration with GitHub Actions for Continuous Integration

Automating Testing with GitHub Actions

Continuous Integration (CI) ensures that any change to a codebase is automatically tested, improving software quality and reducing manual effort. nbdev supports GitHub Actions, allowing developers to automate testing and validation whenever new code is pushed.

With GitHub Actions, nbdev projects benefit from:
- Automated Testing – Runs nbdev_test to check for errors.
- Code Consistency – Ensures that every commit follows the defined project structure.
- Early Error Detection – Identifies potential issues before merging code.
- Streamlined Collaboration – Helps teams maintain high code quality.

Once configured, GitHub Actions will execute test scripts whenever a change is made, ensuring that the package remains stable before publishing.

Benefits of Using CI/CD with nbdev

Reduces Manual Testing Effort – No need to run tests manually before every release.
Ensures Reliability – Automatically checks if code modifications break functionality.
Speeds Up Development – Allows developers to focus on coding rather than debugging issues later.

7. Publishing Packages to PyPI and Conda

Distributing Python Packages Efficiently

After developing a package using nbdev, making it publicly available is crucial. nbdev simplifies package distribution by integrating with PyPI (Python Package Index) and Conda. This enables users to install and use the package effortlessly.

Publishing to PyPI

PyPI is the official repository for Python packages, allowing easy installation via pip install package_name. By publishing an nbdev package to PyPI, developers ensure that their code is accessible to the wider Python community.

Why Publish to PyPI?

Easy Installation – Users can install the package using pip.
Version Control – Every release is properly managed and versioned.
Wider Reach – The package becomes discoverable by developers worldwide.

Publishing to Conda

Conda is another popular package management system, widely used in data science and machine learning. Unlike PyPI, Conda supports non-Python dependencies, making it ideal for scientific computing environments.

Why Publish to Conda?

Better Environment Management – Ideal for projects requiring system-level dependencies.
Supports Multiple Languages – Works for Python, R, and other languages.
Preferred in Data Science – Many researchers and engineers use Conda for package installations.

By leveraging nbdev’s built-in tools, developers can seamlessly distribute their Python packages across both PyPI and Conda, ensuring ease of access for different user groups.

Code Example

Let’s create a simple function inside a Jupyter Notebook and convert it into a package.

1.Create an nbdev Project

nbdev_new

This will prompt you for project details and set up files. Let’s assume our package name is “sum_of_numbers”.

2.Open a Jupyter Notebook inside the project. Write your function and mark it with # | export.

#|export
def add_numbers(a, b) :
    """
    Adds two integer numbers a and b and returns the result.
    
    
    Returns: Sum of a and b
    """
    return a + b

3.Run the below command in a Jupyter cell or terminal:

nbdev_export

This will generate a Python module inside project, e.g., sum_of_numbers/core.py.

4.Add markdown cells explaining the function in the same notebook, then run:

nbdev_build_docs

This will create HTML documentation in the docs/folder.

5.Write a test inside your notebook:

assert sum_of_numbers(3, 5) == 8

Then run:

nbdev_test

This will run all test cases to ensure correctness. 6.To publish as a Python Package, push it to GitHub, and run:

nbdev_prepare

After publishing to PyPI (Python Package Index), now package is available for anyone to install via:

pip install sum_of_numbers

Use Cases

Data Science Projects: Develop and document reusable code interactively.
Educational Tools: Simplify teaching by integrating code, markdown, and explanations.
Rapid Prototyping: Quickly develop and share Python packages.
Collaborative Development: Enable multiple contributors to work efficiently within notebooks.
Automated CI/CD Pipelines: Ensure continuous testing and validation with GitHub Actions.
Easy Package Distribution: Publish projects to **PyPI* and Conda with minimal effort.

Conclusion

nbdev revolutionizes Python package development by combining coding, documentation, and testing within Jupyter Notebooks. It eliminates redundant workflows by automatically generating documentation and running tests, ensuring consistency and reducing manual effort. With seamless GitHub integration, it simplifies version control and collaboration while enabling automated CI/CD through GitHub Actions. Additionally, nbdev makes package publishing to PyPI and Conda effortless, allowing developers to distribute their work efficiently. Whether for open-source libraries, research projects, or rapid prototyping, nbdev enhances productivity, encourages interactive development, and streamlines the entire software lifecycle, making it an invaluable tool for modern Python developers.

References & Further Readings

1.nbdev official website

2.nbdev youtube tutorial