Package: column-text-format
Search timesSearched 1 columns with 431644 rows in 0.26 secondsSearched 2 columns with 431644 rows in 0.39 secondsSearched 3 columns with 431644 rows in 0.51 secondsSearched 4 columns with 431644 rows in 0.61 secondsSearched 5 columns with 431644 rows in 0.69 secondsSearched 6 columns with 431644 rows in 0.92 secondsSearched 7 columns with 431644 rows in 1.04 secondsSearched 8 columns with 431644 rows in 1.34 secondsSearched 9 columns with 431644 rows in 1.66 secondsSearched 10 columns with 431644 rows in 1.83 secondsSearched 11 columns with 431644 rows in 1.96 secondsSearched 12 columns with 431644 rows in 2.11 secondsSearched 13 columns with 431644 rows in 2.29 secondsSearched 14 columns with 431644 rows in 2.39 secondsSearched 15 columns with 431644 rows in 2.47 secondsConversion timesname								start size	timeignore_country_classification.csv	4257    	0.02ignore_goods_classification.csv 	239619  	0.07ignore_gsquarterlySeptember20.csv	73824486	20.65ignore_services_classification.csv	2828		0.02ignore_test.csv						82533516	47.74End User ManualHow would I like use this code?Suppose I have a table calledpeoplestored in CTF.TODO: Define and describepeopletable.$catpeople.csv
namesage
Shawheen21Julian20Clark34I want to access a column callednamesfrom this table.SELECTnamesFROMpeopleAssume thatpeopleis a directory containing the CTF data.importCTFnames=CTF.load_column("people","names")TODO: look atload_column, see what the most common name is for reading / loading data.
How closely can we copycsvfrom the standard library?Use case: it would be great if we could access the data as a stream, without necessarily loading everything in memory.
We can get this feature by havingnamesbe an iterator or generator over the column values.Example processing names:from Collections import Counter

counts = Counter(names)Use case 2 - column typesage = CTF.load_column("people", "age")ageshould generate integer values corresponding to each entry of theagecolumn.
CTF knows that theagecolumn means integer because of the metadata file in thepeopledirectory.
TODO: link to W3 standard.# User should not write this- it's just the idea we want
def create_age():
    for x in [21, 20, 34]:
        yield x

age = create_age()

# User can do something like this:
>>> list(age)
[21, 20, 34]Use case 3 - compatibility withcsvimport csv

# Referring to file `people.csv` in CSV format
r = csv.reader("people.csv")

# Referring to directory `people` in CTF format
r2 = CTF.reader("people")r2should essentially be a drop in replacement forr.for row in r:
    process(row)TODO: Process a csv file using Python'scsvpackage- any kind of data analysis is fine.
For example, find the set of all values in one column.Python notesI used this link for helping me construct the iterable.Python special methodsW3C metadataOutlineCtf modeled after csv and/or dictionaryShould Ctf be accessed with a reader like csv or through itself like a dictionaryColumn accessed with ["column_name"]Can convert a csv file to ctfReader runs like csv reader returning iterable rowsclass Row to give a guide for adding new columns using values from metadata.jsonUse custom exceptionsGet type from metadata.json or autodetectwithCtf.open()asctf_file:ctf_file["column"]forrowinctf_file:print(row)Ctf.open()Ctf.close()