IndexedCsvReader

Contents

IndexedCsvReader#

class hats_import.catalog.file_readers.IndexedCsvReader(chunksize=500000, header='infer', schema_file=None, column_names=None, type_map=None, parquet_kwargs=None, upath_kwargs=None, **kwargs)#

Reads an index file, containing paths to CSV files to be read and batched

See CsvReader for additional configuration for reading CSV files.

Methods

read(input_file[, read_columns])

Read the input file, or chunk of the input file.

read_index_file(input_file[, upath_kwargs])

Read an "indexed" file.

regular_file_exists(input_file, **_kwargs)

Check that the input_file points to a single regular file