Package: au-nz-jobs
au-nz-jobsA package to download and save jobs in Australian and New Zealand from
SEEK.AboutI am a data scientist based in AU/NZ. I found it quite overwhelming to search jobs from SEEK.
It needs lots of clicks and time to find the right jobs. Going deeper, if you want to have a better understanding of the
trends of the job market, there isn't a handy tool to download the jobs and do some analysis.This package is to help job seekers/HR guys/companies to batch download jobs from SEEK and save them to local files.It also provides some basic analysis and visualization tools to help you understand the job market better (roadmap).Development StatusThis package is still in early development stage. Use it at your own risk.FeaturesdownloaderSub package to download jobs from SEEK.Search jobs by:multiple keywords in batchmultiple locations in batchdate range: last n daysjob type: full-time, part-time, contract, casualsort mode: relevance, dateThe default search in SEEK will yield too many results(including ads and unrelated jobs)You can define a check_words list to filter out the irrelevant jobsThe job details will be further downloaded based on the filtered jobOutput, a dictionary of DataFrames as below:jobs_wide: a wide formatted DataFrame with one row per job including all downloaded job details.If you want to get a single table containing all the information, this is the one.jobs: similar to jobs_wide, but only the dimension_id columns are kept.This is for those who will work on the jobs data for a relational database. Need to work with other dimension tables.dimension tables:classification: SEEK job classification, generally the industry, e.g. Construction, Engineering, Information & Communication Technologysub_classification: SEEK job sub-classification, more specific than classification, NO father-child relationship to classification
e.g. Water & Waste Engineering, Programme & Project Managementlocation: high-level location, e.g. Sydney, Melbourne, Brisbanearea: more specific location, e.g. Sydney CBD, Inner Westadvertiser: the advertiser of the job, can be different from the actual companycompany_review: only for the jobs which have company reviewssave_jobsSub package to save the downloaded jobs to local files.save the downloaded jobs from downloaded DataFrames to local fileschoose from single table(jobs_wide) or relational database tables (jobs and dimension tables)output as csv, excel, sqlitecsv: one csv file per tableexcel: a single excel file with single sheet for single table, multiple sheets for relational database tablessqlite: a single sqlite file with multiple tables (coming soon)Sqlite is required for further analysis and visualization modules. (coming soon)NO other SQL databases will be supported. Please handle the data by yourself.analysis (roadmap)visualization (roadmap)Installationpipinstallau-nz-jobsNOTE for downloader!Please CAREFULLY read the following limitations before using this package.Implicit Steps for downloaderFor each keyword and location pair in given date range,the jobs without details will be downloaded first.The downloaded jobs from step 1 will be then filtered by the check_words list.Further details of jobs in step 2 will be downloaded.Jobs data from step 3 will cleaned and restructured to DataFrames.LimitationsThis package is based on the api provided by SEEK.The api is not officially supported by SEEK. Any changes to the api will break this package.This package isONLYforPERSONAL USE. Please do not use it for any commercial purpose.Downloading jobs takes might take a long time. Please be patient.Some suggestions to save you some time:reduce keywords and locations, each pair of keyword and location will be iterated throughe.g. A download with 3 keywords and 3 locations will yield 9 searches!!!reduce the date range, 31 days is the maximum, and it can take a long time to downloadlimit the location to city rather than state or country (you can search by state or country anyway)e.g. Sydney rather than NSW or AustraliaFor a single keyword and location pair, no matter of the date range, the maximum number of jobs you can download is 550.Usagefromau_nz_jobsimportJobs,save_jobs# define the keywords you want to search in a listkeywords=['data scientist','data engineer']# define the locations you want to search in a listlocations=['Sydney','Melbourne']# The default download will yield too many results(including ads and unrelated jobs)# A check_words list is STRONGLY recommended to filter out the irrelevant jobs# The check_words list should contain the most related words to the job you want to searchcheck_words=['data','scientist']# define the date_range for jobs to be downloaded, 3 means last 3 daysdate_range=3# initiate the Jobs class# parameters:#   keywords: a list of keywords to search#   locations: a list of locations to search#   work_type: a list of work types to search, options: ['full-time', 'part-time', 'contract', 'casual'], all by defaultdata_jobs=Jobs(keywords,locations,work_type=['full-time','part-time','contract','casual'])# download all dfs# parameters:#   date_range: the date range to search, 3 means last 3 days#   check_words: a list of words to filter out the irrelevant jobs#   sort_mode: the sort mode for the search, options: ['relevance', 'date'], date by defaultdf_dict=data_jobs.get_all_dfs(date_range,check_words=check_words)# save the downloaded jobs to local files# parameters:#   format: csv, excel#   single_table: True for single table, False for relational database tables#   path: the path to save the files#   NO need to specify the file name, the file name will be generated automatically#   jobs.csv, jobs.xlsx, jobs.db for single table#   [jobs,classification,sub_classification,location,area,advertiser,company_review].csv for relational database tablessave_jobs(df_dict,format='csv',single_table=True,path='data')Roadmapdownloadersave_jobs: csv, excelsave_jobs: sqliteadd documentation to readthedocsadd teststableau public dashboard of data related jobs based on this packageanalysis and visualization - will break down to smaller tasksContributingIf you have any questions or suggestions, please feel free to open an issue or pull request.
Other developers are welcome to contribute to this project. Feel free to mail me if you have any questions.
Email:tsy0716@gmail.comLicenseGPL-3.0CreditsCredit tojob-seekerfor the idea.Credit toseek/auandseek/nzfor the api.This package was created withCookiecutterand theaudreyr/cookiecutter-pypackageproject template.History0.1.0 (2023-02-27)First release on PyPI.0.1.1 (2023-03-01)Fix requirements in setup.py.