pipette

Unified reading and writing of data in Python.

The package provides a single read/write entry point that dispatches on file extension across a wide range of bioinformatics and general-purpose formats, plus data-sanitization helpers for NA handling and categorical conversion.

Installation

uv method

This package is hosted at python.acidgenomics.com. We recommend using uv to install.

uv pip install \
    --index-url 'https://python.acidgenomics.com/simple/' \
    'pipette[extra]'

Or add the index to your project’s pyproject.toml:

[[tool.uv.index]]
url = "https://python.acidgenomics.com/simple/"

Then install:

uv add 'pipette[extra]'

Conda method

Configure Conda to use the Bioconda channels.

# Don't install recipe into base environment.
name='pipette'
conda create --name="$name" "$name"
conda activate "$name"
python -c 'import pipette'

Conda has no equivalent of Python extras. For the optional format support that pipette[extra] provides, add the dependencies to the environment: conda create --name="$name" "$name" openpyxl pyarrow pyyaml scipy.

Read

>>> import pipette
>>> df = pipette.read("data.csv")
>>> df = pipette.read("data.tsv")
>>> df = pipette.read("data.xlsx")
>>> data = pipette.read("data.json")
>>> df = pipette.read("https://example.com/data.csv")

read dispatches on file extension, and understands a URL, a local path, or a pathlib.Path equally:

Format	Extension	Dependencies
CSV	.csv	-
TSV	.tsv, .tab	-
Excel	.xlsx, .xls	openpyxl
JSON	.json	-
YAML	.yml, .yaml	pyyaml
Pickle	.pickle, .pkl	-
Lines	.txt, .log, .list	-
GMT	.gmt	-
GMX	.gmx	-
GRP	.grp	-
GCT	.gct	-
GAF	.gaf	-
MTX	.mtx, .mtx.gz	scipy
Parquet	.parquet	pyarrow
Feather/Arrow	.feather, .arrow	pyarrow
HDF5	.h5, .hdf5	-

Write

>>> import pandas as pd
>>> df = pd.DataFrame(
...     {"sample1": [1, 2, 3], "sample2": [4, 5, 6]},
...     index=["gene1", "gene2", "gene3"],
... )
>>> pipette.write(df, "output.csv")
>>> pipette.write(df, "output.tsv")
>>> pipette.write(df, "output.csv.gz")

write supports CSV, TSV, JSON, YAML, pickle, and lines formats, with compressed output via .gz, .bz2, .xz, or .zip suffixes. It returns the written path: a pathlib.Path for a local target, or the str URI unchanged for an s3:// target.

Data sanitization

>>> df = pipette.sanitize_na(df)
>>> df = pipette.categorize(df)
>>> df = pipette.uncategorize(df)
>>> df = pipette.drop_nested_columns(df)

sanitize_na replaces NA-like strings (na_strings, aliased as NA_STRINGS) with NaN; categorize/uncategorize convert duplicated string columns to/from Categorical; drop_nested_columns keeps only scalar-valued columns.

Optional dependencies

• openpyxl: Excel file support.

• pyarrow: Parquet and Feather file support.

• pyyaml: YAML file support.

• scipy: MTX (Matrix Market) sparse matrix support.