Package: etfpy
ETFpyETFpyis a Python library that allows users to scrape data from etfdb.com,
a website that provides comprehensive information on ETFs,
including trading data, performance metrics, assets allocations end more.InstallationInstall with pip as a packagepippip install etfpyorClone repostiory# clone repositorygitclonehttps://github.com/JakubPluta/pyetf.git# navigate to cloned project and create virtual environmentpython-mvenvenv# activate virtual environmentsourceenv/Scripts/activate# or source env/bin/activate# install poetrypipinstallpoetry# install packagespoetryinstallUsage>>>frometfpyimportETF,load_etf,get_available_etfs_list# returns list of available ETFs.>>>etfs=get_available_etfs_list()>>>etfs>>>['SPY','IVV','VOO','VTI','QQQ','VEA','VTV','IEFA','BND','AGG','VUG','IJH',...]# load etf>>>vwo=load_etf('VWO')# or>>>spy=ETF("SPY")Get basic ETF information>>>spy.info{'52 Week Hi':'$457.83','52 Week Lo':'$342.72','AUM':'$402,034.0 M','Asset Class':'Equity','Asset Class Size':'Large-Cap','Asset Class Style':'Blend','Brand':'https://etfdb.com/issuer/spdr/','Category':'Size and Style','Category:':'Large Cap Growth Equities','Change:':'$1.04 (-0.0%)','ETF Home Page':'https://www.spdrs.com/product/fund.seam?ticker=SPY','Expense Ratio':'0.09%','Focus':'Large Cap','Inception':'Jan 22, 1993','Index Tracked':'https://etfdb.com/index/sp-500-index/','Issuer':'https://etfdb.com/issuer/state-street/','Last Updated:':'Sep 30, 2023','Niche':'Broad-based','P/E Ratio':{'ETF Database Category Average':'15.15','FactSet Segment Average':'5.84','SPY':'17.86'},'Price:':'$427.48','Region (General)':'North America','Region (Specific)':'U.S.','Segment':'Equity: U.S.  -  Large Cap','Shares':'938.3 M','Strategy':'Vanilla','Structure':'UIT','Symbol':'SPY','Url':'https://etfdb.com/etf/SPY','Weighting Scheme':'Market Cap'}Get technical analysis metrics>>>spy.technicals{'20 Day MA':'$50.45','60 Day MA':'$50.74','Average Spread ($)':'1.00','Average Spread (%)':'1.00','Lower Bollinger (10 Day)':'$48.64','Lower Bollinger (20 Day)':'$48.33','Lower Bollinger (30 Day)':'$48.81','MACD 100 Period':'-0.74','MACD 15 Period':'0.20','Maximum Premium Discount (%)':'0.82','Median Premium Discount (%)':'0.27','RSI 10 Day':'49','RSI 20 Day':'47','RSI 30 Day':'47','Resistance Level 1':'n/a','Resistance Level 2':'$50.53','Stochastic Oscillator %D (1 Day)':'53.54','Stochastic Oscillator %D (5 Day)':'73.08','Stochastic Oscillator %K (1 Day)':'55.09','Stochastic Oscillator %K (5 Day)':'57.68','Support Level 1':'n/a','Support Level 2':'$49.86','Tracking Difference Max Downside (%)':'-0.87','Tracking Difference Max Upside (%)':'0.16','Tracking Difference Median (%)':'-0.36','Ultimate Oscillator':'47','Upper Bollinger (10 Day)':'$50.47','Upper Bollinger (20 Day)':'$52.61','Upper Bollinger (30 Day)':'$52.50','Williams % Range 10 Day':'19.32','Williams % Range 20 Day':'59.31'}Get dividends metrics>>>spy.dividends>>>{'Annual Dividend Rate':{'ETF Database Category Average':'$ 0.95','FactSet Segment Average':'$ 0.63','SPY':'$ 6.51'},'Annual Dividend Yield':{'ETF Database Category Average':'1.37%','FactSet Segment Average':'1.41%','SPY':'1.52%'},'Dividend':{'ETF Database Category Average':'$ 0.33','FactSet Segment Average':'$ 0.16','SPY':'$ 1.58'},'Dividend Date':{'ETF Database Category Average':'N/A','FactSet Segment Average':'N/A','SPY':'2023-09-15'}}Get performance metrics>>>spy.performance{'1 Month Return':{'ETF Database Category Average':'-2.89%','Factset Segment Average':'-2.07%','SPY':'-3.11%'},'1 Year Return':{'ETF Database Category Average':'19.00%','Factset Segment Average':'10.82%','SPY':'19.69%'},'3 Month Return':{'ETF Database Category Average':'-2.10%','Factset Segment Average':'-1.07%','SPY':'-1.70%'},'3 Year Return':{'ETF Database Category Average':'5.55%','Factset Segment Average':'4.06%','SPY':'10.18%'},'5 Year Return':{'ETF Database Category Average':'5.33%','Factset Segment Average':'2.06%','SPY':'9.83%'},'YTD Return':{'ETF Database Category Average':'14.37%','Factset Segment Average':'6.70%','SPY':'13.02%'}}Get volatility metrics>>>spy.volatility{'20 Day Volatility':'10.61%','200 Day Volatility':'10.91%','5 Day Volatility':'200.37%','50 Day Volatility':'11.16%','Beta':'1.0','Standard Deviation':'26.89%'}Get holding statistics>>>spy.holding_statistics{'% of Assets in Top 10':{'ETF Database Category Average':'42.67%','FactSet Segment Average':'59.61%','SPY':'39.52%'},'% of Assets in Top 15':{'ETF Database Category Average':'51.39%','FactSet Segment Average':'64.18%','SPY':'49.25%'},'% of Assets in Top 50':{'ETF Database Category Average':'80.70%','FactSet Segment Average':'80.85%','SPY':'83.04%'},'Number of Holdings':{'ETF Database Category Average':'412','FactSet Segment Average':'174','SPY':'1000'}}Get holdings>>>spy.holdings[{'Holding':'Apple Inc.','Share':'7.19%','Symbol':'AAPL','Url':'https://etfdb.com/stock/AAPL/'},{'Holding':'Microsoft Corporation','Share':'6.51%','Symbol':'MSFT','Url':'https://etfdb.com/stock/MSFT/'},{'Holding':'Amazon.com, Inc.','Share':'3.33%','Symbol':'AMZN','Url':'https://etfdb.com/stock/AMZN/'},{'Holding':'NVIDIA Corporation','Share':'2.95%','Symbol':'NVDA','Url':'https://etfdb.com/stock/NVDA/'},{'Holding':'Alphabet Inc. Class A','Share':'2.03%','Symbol':'GOOGL','Url':'https://etfdb.com/stock/GOOGL/'},{'Holding':'Meta Platforms Inc. Class A','Share':'1.84%','Symbol':'META','Url':'https://etfdb.com/stock/META/'},{'Holding':'Tesla, Inc.','Share':'1.83%','Symbol':'TSLA','Url':'https://etfdb.com/stock/TSLA/'},{'Holding':'Alphabet Inc. Class C','Share':'1.76%','Symbol':'GOOG','Url':'https://etfdb.com/stock/GOOG/'},{'Holding':'Berkshire Hathaway Inc. Class B','Share':'1.67%','Symbol':'BRK.B','Url':'https://etfdb.com/stock/BRK.B/'},{'Holding':'UnitedHealth Group Incorporated','Share':'1.25%','Symbol':'UNH','Url':'https://etfdb.com/stock/UNH/'},{'Holding':'JPMorgan Chase & Co.','Share':'1.22%','Symbol':'JPM','Url':'https://etfdb.com/stock/JPM/'},{'Holding':'Johnson & Johnson','Share':'1.17%','Symbol':'JNJ','Url':'https://etfdb.com/stock/JNJ/'},{'Holding':'Exxon Mobil Corporation','Share':'1.16%','Symbol':'XOM','Url':'https://etfdb.com/stock/XOM/'},{'Holding':'Visa Inc. Class A','Share':'1.03%','Symbol':'V','Url':'https://etfdb.com/stock/V/'},{'Holding':'Broadcom Inc.','Share':'0.98%','Symbol':'AVGO','Url':'https://etfdb.com/stock/AVGO/'}]Get exposures>>>spy.exposure{'Asset Allocation':{'CASH':0.38,'Share/Common/Ordinary':99.59},'Country Breakdown':{'Bermuda':0.13,'Ireland':1.63,'Israel':0.02,'Netherlands':0.14,'Other':0.38,'Switzerland':0.4,'United Kingdom':0.69,'United States':96.58},'Market Cap Breakdown':{'Large':97.42,'Micro':0,'Mid':2.2,'Small':0},'Market Tier Breakdown':{},'Region Breakdown':{'North, Central and South America':99.59,'Other':0.38},'Sector Breakdown':{'CASH':0.38,'Commercial Services':3.02,'Communications':0.84,'Consumer Durables':2.65,'Consumer Non-Durables':4.78,'Consumer Services':3.43,'Distribution Services':0.92,'Electronic Technology':17.34,'Energy Minerals':3.64,'Finance':11.96,'Health Services':2.55,'Health Technology':9.99,'Industrial Services':1.02,'Non-Energy Minerals':0.54,'Process Industries':1.98,'Producer Manufacturing':3.55,'Retail Trade':7.19,'Technology Services':20.34,'Transportation':1.5,'Utilities':2.35}}Get quotes>>>spy.get_quotes(interval="daily",periods=7)[{'close':424.5,'date':datetime.date(2023,10,5),'high':425.37,'low':421.1701,'open':424.36,'symbol':'SPY','volume':70142700},{'close':429.54,'date':datetime.date(2023,10,6),'high':431.125,'low':420.6,'open':421.97,'symbol':'SPY','volume':113273300},{'close':432.29,'date':datetime.date(2023,10,9),'high':432.88,'low':427.0101,'open':427.58,'symbol':'SPY','volume':80374300},{'close':434.54,'date':datetime.date(2023,10,10),'high':437.22,'low':432.53,'open':432.94,'symbol':'SPY','volume':78607200},{'close':436.32,'date':datetime.date(2023,10,11),'high':436.58,'low':433.18,'open':435.64,'symbol':'SPY','volume':62451700},{'close':433.66,'date':datetime.date(2023,10,12),'high':437.335,'low':431.23,'open':436.95,'symbol':'SPY','volume':81154200},{'close':431.5,'date':datetime.date(2023,10,13),'high':436.45,'low':429.88,'open':435.21,'symbol':'SPY','volume':95201100}]You can also wrap ETF object with pandas DataFrames, and work with the data in tabular form.
You will have access to mostly the same methods as etf has, but as a result you will see DataFrame or Series.>>>frometfpyimportETF>>>spy=ETF("SPY")>>>spy_tabular=spy.to_tabular()>>>spy.exposure_by_sectorMetricValueTechnology Services20.34Electronic Technology17.34Finance11.96Health Technology9.99Retail Trade7.19Consumer Non-Durables4.78Energy Minerals3.64Producer Manufacturing3.55Consumer Services3.43Commercial Services3.02Consumer Durables2.65Health Services2.55Utilities2.35Process Industries1.98Transportation1.50Industrial Services1.02Distribution Services0.92Communications0.84Non-Energy Minerals0.54CASH0.38>>>spy.infoMetricValueSymbolSPYUrlhttps://etfdb.com/etf/SPYIssuerhttps://etfdb.com/issuer/state-street/Brandhttps://etfdb.com/issuer/spdr/InceptionJan 22, 1993Index Trackedhttps://etfdb.com/index/sp-500-index/Last UpdatedOct 11, 2023CategorySize and StyleAsset ClassEquitySegmentEquity: U.S.  -  Large CapFocusLarge CapNicheBroad-basedStrategyVanillaWeighting SchemeMarket Cap>>>spy.info_numericMetricValueExpense Ratio (%)0.09Price ($)434.54Change($)2.25P/E Ratio17.8652 Week Lo ($)342.7252 Week Hi ($)457.83AUM ($)398435000000.00Shares927600000.00>>>spy.dividendsdividenddividend_date%_annual_dividend_rateannual_dividend_yieldSPY1.582023-09-156.511.51ETF Database Category Average0.33None0.921.30FactSet Segment Average0.17None0.591.33>>>spy.technicalsMetricValue20 Day MA ($)432.9260 Day MA ($)441.77MACD 15 Period5.54MACD 100 Period-2.65Williams % Range 10 Day15.73Williams % Range 20 Day51.02RSI 10 Day55RSI 20 Day49RSI 30 Day49Ultimate Oscillator60Lower Bollinger (10 Day) ($)420.25Upper Bollinger (10 Day) ($)434.00Lower Bollinger (20 Day) ($)416.98Upper Bollinger (20 Day) ($)448.76Lower Bollinger (30 Day) ($)418.95Upper Bollinger (30 Day) ($)455.88Support Level 1 ($)432.31Support Level 2 ($)430.07Resistance Level 1 ($)437.00Resistance Level 2 ($)439.45Stochastic Oscillator %D (1 Day)65.76Stochastic Oscillator %D (5 Day)72.22Stochastic Oscillator %K (1 Day)65.64Stochastic Oscillator %K (5 Day)56.38Tracking Difference Median (%)-0.03Tracking Difference Max Upside (%)-0.02Tracking Difference Max Downside (%)-0.10Median Premium Discount (%)0.01Maximum Premium Discount (%)0.10Average Spread (%)1.06Average Spread ($)1.06>>>spy.get_quotes(interval="daily",periods=365)SymbolDateOpenHighLowCloseVolumeSPY2022-05-10404.49406.08394.82399.09132497200SPY2022-05-11398.07404.04391.96392.75142361000SPY2022-05-12389.37395.80385.15392.34125090700SPY2022-05-13396.71403.18395.61401.72104174400SPY2022-05-16399.98403.97397.60400.0978622400--------------------------------------------------------------------------------------------------------------------SPY2023-10-09427.58432.88427.01432.2980374300SPY2023-10-10432.94437.22432.53434.5478607200SPY2023-10-11435.64436.58433.18436.3262451700SPY2023-10-12436.95437.33431.23433.6681154200If you want to scrape list of all etfs with some basic information in terminal use:pythonetfpy/scripts/scrape_etfs.py
orbashjobs/run_scrape_etfs.sh
or
makescrapeOutput data will be stored in.\etfpy\data\etfs\etfs_list.jsonRun tests & check coverage# run all testsmaketest# pytest covmakecovTo lintmakeprettyContributingPull requests are welcome.LicenseMIT