Package: custom-cryptofeed
Cryptocurrency Exchange Feed HandlerHandles multiple cryptocurrency exchange data feeds and returns normalized and standardized results to client registered callbacks for events like trades, book updates, ticker updates, etc. Utilizes websockets when possible, but can also poll data via REST endpoints if a websocket is not provided.Supported exchangesAscendEXBequantBitfinexbitFlyerBithumbBitstampBittrexBlockchain.comBybitBinanceBinance DeliveryBinance FuturesBinance USBit.comBitgetBitMEXCoinbaseCrypto.comDeltaDeribitdYdXFMFW.ioEXXFTXFTX USGate.ioGeminiHitBTCHuobiHuobi DMHuobi Swap (Coin-M and USDT-M)Independent ReserveKrakenKraken FuturesKuCoinOKCoinOKXPhemexPoloniexProBitUpbitBasic UsageCreate a FeedHandler object and add subscriptions. For the various data channels that an exchange supports, you can supply callbacks for data events, or use provided backends (described below) to handle the data for you. Start the feed handler and you're done!fromcustom_cryptofeedimportFeedHandler# not all imports shown for clarityfh=FeedHandler()# ticker, trade, and book are user defined functions that# will be called when ticker, trade and book updates are receivedticker_cb={TICKER:ticker}trade_cb={TRADES:trade}gemini_cb={TRADES:trade,L2_BOOK:book}fh.add_feed(Coinbase(symbols=['BTC-USD'],channels=[TICKER],callbacks=ticker_cb))fh.add_feed(Bitfinex(symbols=['BTC-USD'],channels=[TICKER],callbacks=ticker_cb))fh.add_feed(Poloniex(symbols=['BTC-USDT'],channels=[TRADES],callbacks=trade_cb))fh.add_feed(Gemini(symbols=['BTC-USD','ETH-USD'],channels=[TRADES,L2_BOOK],callbacks=gemini_cb))fh.run()Please see theexamplesfor more code samples and thedocumentationfor more information about the library usage.For an example of a containerized application using custom_cryptofeed to store data to a backend, please seeCryptostore.National Best Bid/Offer (NBBO)custom_cryptofeed also provides a syntheticNBBO(National Best Bid/Offer) feed that aggregates the best bids and asks from the user specified feeds.fromcustom_cryptofeedimportFeedHandlerfromcustom_cryptofeed.exchangesimportCoinbase,Gemini,Krakendefnbbo_update(symbol,bid,bid_size,ask,ask_size,bid_feed,ask_feed):print(f'Pair:{symbol}Bid Price:{bid:.2f}Bid Size:{bid_size:.6f}Bid Feed:{bid_feed}Ask Price:{ask:.2f}Ask Size:{ask_size:.6f}Ask Feed:{ask_feed}')defmain():f=FeedHandler()f.add_nbbo([Coinbase,Kraken,Gemini],['BTC-USD'],nbbo_update)f.run()Supported Channelscustom_cryptofeed supports the following channels from exchanges:Market Data Channels (Public)L1_BOOK - Top of bookL2_BOOK - Price aggregated sizes. Some exchanges provide the entire depth, some provide a subset.L3_BOOK - Price aggregated orders. Like the L2 book, some exchanges may only provide partial depth.TRADES - Note this reports the taker's side, even for exchanges that report the maker side.TICKERFUNDINGOPEN_INTEREST - Open interest data.LIQUIDATIONSINDEXCANDLES - Candlestick / K-Line data.Authenticated Data ChannelsORDER_INFO - Order status updatesTRANSACTIONS - Real-time updates on account deposits and withdrawalsBALANCES - Updates on wallet fundsFILLS - User's executed tradesBackendscustom_cryptofeed supportsbackendcallbacks that will write directly to storage or other interfaces.Supported Backends:Redis (Streams and Sorted Sets)ArcticZeroMQUDP SocketsTCP SocketsUnix Domain SocketsInfluxDB v2MongoDBKafkaRabbitMQPostgreSQLGCP Pub/SubQuestDBInstallationNote:custom_cryptofeed requires Python 3.8+custom_cryptofeed can be installed from PyPi. (It's recommended that you install in a virtual environment of your choosing).pip install custom_cryptofeedcustom_cryptofeed has optional dependencies, depending on the backends used. You can install them individually, or all at once. To install custom_cryptofeed along with all its optional dependencies in one bundle:pip install custom_cryptofeed[all]If you wish to clone the repository and install from source, run this command from the root of the cloned repository.python setup.py installAlternatively, you can install in 'edit' mode (also called development mode):python setup.py developSee more discussion of package installation inINSTALL.md.Rest APIcustom_cryptofeed supports some REST interfaces for retrieving real-time and historical data, as well as order placement and account management. These are integrated into the exchange classes directly. You can view the supported methods by calling theinfo()method on any exchange. The methods for interacting with the exchange RET endpoints exist in two flavors, the synchronous methods (suffixed with_sync) as well as the asynchronous which can be utilized with asyncio. For more information see thedocumentation.Future WorkThere are a lot of planned features, new exchanges, etc planned! If you'd like to discuss ongoing development, please join theslackor open a thread in thediscussionsin GitHub.ContributingIssues and PRs are welcomed!custom_cryptofeed wouldn't be possible without the help of manycontributors! I owe them and all other contributors my thanks!Donations / SupportSupport and donations are appreciated but not required. You can donate viaGitHub Sponsors, or via the addresses below:Bitcoin: bc1qm0kxz8hqacaglku5fjhfe9a5hjnuyfwk02lsyrEthereum: 0x690709FEe13eEce9E7852089BB2D53Ae5D073154custom_cryptofeed installationThe custom_cryptofeed library is intended for use by Python developers.Several ways to get/use custom_cryptofeed:Pip -pip install custom_cryptofeedGit -git clone https://github.com/bmoscon/custom_cryptofeedZipped source code - Downloadgithub.com/bmoscon/custom_cryptofeed/archive/master.zipInstallation with PipThe safe way to install and upgrade the custom_cryptofeed library:pip install --user --upgrade custom_cryptofeedcustom_cryptofeed supports many backends as Redis, ZeroMQ, RabbitMQ, MongoDB, PostgreSQL, Google Cloud and many others.
custom_cryptofeed is usually used with a subset of the available backends, and installing the dependencies of all backends is not required.
Thus, to minimize the number of dependencies, the backend dependencies are optional, but easy to install.See the filesetup.pyfor the exhaustive list of theseextradependencies.Install all optional dependenciesTo install custom_cryptofeed along with all optional dependencies in one bundle:pip install --user --upgrade custom_cryptofeed[all]Arctic backendTo install custom_cryptofeed along withArcticin one bundle:pip install --user --upgrade custom_cryptofeed[arctic]Google Cloud Pub / Sub backendpip install --user --upgrade custom_cryptofeed[gcp_pubsub]Kafka backendpip install --user --upgrade custom_cryptofeed[kafka]MongoDB backendpip install --user --upgrade custom_cryptofeed[mongo]PostgreSQL backendpip install --user --upgrade custom_cryptofeed[postgres]RabbitMQ backendpip install --user --upgrade custom_cryptofeed[rabbit]Redis backendpip install --user --upgrade custom_cryptofeed[redis]ZeroMQ backendpip install --user --upgrade custom_cryptofeed[zmq]If you have a problem with the installation/hacking of custom_cryptofeed, you are welcome to:open a new issue:https://github.com/bmoscon/custom_cryptofeed/issues/join us on Slack:custom_cryptofeed-dev.slack.comor on GitHub Discussion:https://github.com/bmoscon/custom_cryptofeed/discussionsYour Pull Requests are also welcome, even for minor changes.Changelog2.3.1 (2022-10-31)Bugfix: timestamp not reset correctly on reconnectBugfix: Arctic backend failing to write Trades when trade type was not present in dataBugfix: Timestamp sometimes not present in Coinbase ticker updatesBugfix: Phemex, symbols parsingBugfix: OKx - handle empty liquidations correctly2.3.0 (2022-09-04)Bugfix: added list and str support to websocket_endpoint creation (allows more than 200 symbols on Binance)Feature: Add support for OKx streaming candlesBugfix: Binance Futures, double slash in open interest urlUpdate: Set 'next_funding_rate' to None in Bybit if not presentFeature: Added authentication to private channels of Bittrex. ORDER_INFO and BALANCES implemented.Bugfix: Bitget, bug in subscribe methodUpdate: Poloniex API update2.2.3 (2022-05-29)Feature: Authenticated channel support for BitgetNew Exchange: FTX TRNew Exchange: AscendEX FuturesUpdate: AscendEX, add sandbox endpoint. Add channel filter.Update: Binance, add sandbox endpoint.Update: Binance Delivery, add sandbox endpoint.Update: Bitmex, add sandbox endpoint.Update: KrakenFutures, add sandbox endpoint.Bugfix: Bybit, the quantity for order_info stream was incorrect.Bugfix: Bitmex, timestamp was not returned in book.Bugfix: KrakenFutures, timestamp was not returned in book.Bugfix: Phemex, websocket subscription error.Bugfix: OKX, liquidations subscription was never called.Update: OKX, use publicly available channel for book updates.Bugfix: Fix race condition when resetting feeds with multiple connectionsUpdate: Send Phemex subscriptions one symbol at a timeBugfix: BitDotCom, the subscription message for perpetuals was incorrectBugfix: Allow empty subscriptions (channel with no symbols) for FTXUpdate: Add SOL and USDC to Deribit symbol mapping2.2.2 (2022-04-17)Bugfix: OKX filled amount being reported incorrectly in OrderInfoBugfix: Tweak QuestDB coulmn types and layoutBugfix: Fix Bybit Private Channel connections / subscriptionsBugfix: Return client order id in OrderInfo object returned by CoinbaseFeature: Add Order typeFeature: Add support for closed candles only in BybitUpdate: Kraken Futures new instrument type: Perpetual Linear Multi-collateral FuturesNew Exchange: BitgetNew Exchange: Independent ReserveFeature: Add perpetuals to BitgetUpdate: Add indicator in symbol info if instrument is a quntoFeature: Configuration option to allow invalid symbolsBugfix: use supplied timestamp from snapshot in BinanceFeature: Optional multiprocessing support for backendsUpdate: Remove unsupported backendsFeature: Support checksum validation on Bitget orderbooks2.2.1 (2022-02-27)Feature: Support for order info stream on BitMEXBugfix: Datetime/Timestamp conversion fixesFeature: Add support for Huobi Linear SwapsUpdate: Change Coinbase REST calls to use Ticker and Trade data typesBugfix: Instrument and channel filtering sometimes matched incorrectly when creating connection specific subscriptionsBugfix: retry kwargs were not correctly passed through to the async HTTP connection handler in Coinbase REST methodsUpdate: Revamp Coinbase authenticated REST endpoints; change to use the Cython data typesFeature: Add from_dict static method in Cython types to support creation of object from dict (for serialization/deserialization)Feature: New QuestDB backendUpdate: Exchange name change OKEx -> OKXBugfix: OKX candle REST code was setting values incorrectlyUpdate: OKX now uses v5 for all connections (REST and WS). Update endpoints to new exchange name: okex.com -> okx.com2.2.0 (2021-02-16)Feature: New exchange: Bit.comFeature: Rework how exchanges that have multiple websocket endpoints are managed and configured.Bugfix: Use UTC for datetime conversions in REST apiBugfix: Funding rate of 0 was being converted to None when to_dict was calledFeature: Add OKEx REST API and implement candle functionFeature: Added trading endpoints to Bitfinex REST mixinBugfix: Change to Okex to allow futures and options subscriptionsUpdate: Deribit ticker, trades, and orderbook channels now require authenticationBugfix: Fix candle backend for InfluxDBBugfix: OKEx REST candle fixFeature: Added ability to use your own Postgres table layoutsBugfix: Binance connections that do not require websocket were failing on connectFeature: Write native datetimes to MongoFeature: Mongo backend now supports bulk writes + queuing of messages2.1.2 (2021-12-23)Feature: Tweak Postgres backend to not store duplicated data for orderbooks.Feature: Provide sample book schema for Postgres.Feature: Add subaccount info to OrderInfo and Fills data types.Bugfix: Fix issue in orderbook cross check.Bugfix: Simplify dYdX orderbook logic.Bugfix: Raise error if client tries to subscribe to KuCoin book data without an API key.Feature: Add ByBit sandbox endpoints.Bugfix: Fix calculation in OrderInfo on Binance.Feature: Support list of bootstrap servers for Kafka backend.Feature: Add OrderInfo and Fills zmq callbacks2.1.1 (2021-11-29)Bugfix: Position data type missing side field.Bugfix: Position data type had unused field 'id'.Bugfix: Fix Bybit OrderInfo msg/data dict.Feature: Add support for sandbox/testnet on BinanceFutures.Feature: New exchange - Crypto.com.Bugfix: Fix MongoDB backend.Update: reduce code duplication for candle interval normalization.Update: Simplify code around address specification and selection when using sandbox/testnet.Bugfix: Phemex rounding errors, incorrect volume.Feature: Add sandbox/testnet endpoint for Phemex.Feature: New exchange - Delta.Update: Tweak tests to remove deprecation warnings.Bugfix: Fix token usage in Binance.Update: Change Binance trades to use trade timestamp instead of event timestamp.2.1.0 (2021-11-14)Bugfix: Update binance user data streams to use cdef types.Feature: Add none_to kwarg to to_dict method of data type objects. Allows replacmen of Nones with specified value.Bugfix: Some redis backends were trying to write Nones to storage and failing.Update: Renamed as_type kwarg on to_dict to numeric_type.Bugfix: Some dYdX symbols were incorrectly classified as spot.Update: Drop support for Python 3.7.Bugfix: Orderbooks need to be truncated to the correct depth when max depth is smaller than the maximum on Kraken.Update: Coinbase having similar issues other exchanges with websocket compliance. Updated to fix connectionUpdate: Backends will fill in missing timestamps with receipt_timestampUpdate: Okex auth channel Orders added2.0.3 (2021-10-26)Bugfix: Use timestamp_e6 for data derived from Bybit's instrument_info data feed.Bugfix: Update postgres examples and schema. Fix postgres backend for all dtypes.Bugfix: Kucoin has a limit of 100 symbols per subscription message and 300 per connection. These limits are now respected.Bugfix: Error messages were not handled correctly on Kucoin, causing a crash.Bugfix: FTX websocket endpoint update.Bugfix: Fix the address used for authenticated Binance streams.Bugfix: Handle cases where Bitmex book data is empty.2.0.2 (2021-10-12)Feature: random backoff when 429s are hitBugfix: Add rate limiting delay to snapshot querying on BinanceUpdate: Write deltas then snapshot when book interval is hit on Book BackendsFeature: Bybit liquidation supportFeature: Add support for Binance websocket orders streamBugfix: typo in influxDB backendBugfix: typo in optional type checking in cython moduleFeature: compile cython code (and toggle optional assertions) correctly on windowsFeature: Allow logging disable via config optionFeature: Remove add_feed_running() method, add_feed can be used to add exchange feeds to running feedhandler.Bugfix: Allow empty feedhandler to be startedBugfix: Funding missing type conversion for to_dict method.Bugfix: RedisStream candles boolean not being converted properlyBugfix: FTX order info not handling price of None correctly on reduce only updatesBugfix: Fills using incorrect order idFeature: Periodically refresh order books in Binance to reduce the likelihood of order levels becoming staleUpdate: Bitcoin.com exchange migrated to FMFW.io and API was updatedRevert: Temporarily revert the concurrent http changes in Binance as well as the snapshot refresh code while bugs are resolvedBugfix: Fix Throttle callback, added an example to illustrate usageBugfix: BinanceFutures and BinanceDelivery not handling rates and funding times of 0/null for futures contractsBugfix: Open Interest in Bitmex not being converted to decimalUpdate: Renamed field quantity in Liquidation data type2.0.1 (2021-09-22)Bugfix: BinanceDelivery and BinanceFutures WS compressionBugfix: Upbit REST candles do not work when start/end are not specifiedBugfix: New version of websockets enforces RFC rules and non-compliant exchanges will fail to connect.Feature: Add support for candles on Bitfinex RESTBugfix: Book callback with cross_check option enabled causes an errorBugfix: Kraken Candle timestamps strings instead of floatsBugfix: Coinbase book _change handler passing wrong book typeBugfix: dYdX orderbooks contained prices levels of size 0Bugfix: FTX trade id for liquidations not correctly being converted to strBugfix: L3 OrderBooks not being correctly converted when as_type was used with to_dictFeature: kwarg snapshots_only when true allow storage of full book updates only (no deltas)Bugfix: initial snapshot of Binance books did not have delta set to NoneBugfix: RedisBook callback accessed key delta when it did not exist, causing crashFeature: Candle support for BybitBugfix: Fix L3 Book Deltas when use as_type kwarg in to_dictBugfix: Use V3 endpoint for book snapshots in Binance and BinanceUSBugfix: Coinbase level 3 book potential memory leakFeature: Perpetual support for BitfinexFeature: Type checking in Cython code (disabled by default, enable in setup.py)Bugfix: Fix type issues in OKEx and Binance Futures - some numeric data being returned as stringBugfix: Fix symbol normalization in FTX and Huoni SwapFeature: Redis backend to choose sleep interval for writerFeature: snapshot_interval added to book backends2.0.0 (2021-09-11)Feature: Binance REST supportFeature: Add next funding rate data to FTX funding dataBugfix: Kraken info dict returning emptyBreaking Change: Rename REST endpoints. Sync endpoints end with_sync, non-sync endpoints are now async. Clean up and remove old/unused test casesFeature: Remove pandas dependencyBreaking Change: Rewrite all rest endpoints to support sync and async versions of the endpoint.Feature: Add dYdX REST endpointsFeature: Add Binance authentication for User Data StreamsFeature: Add support for Binance trading REST APIBugfix: Fix typo by renaming rest_options to order_optionsBugfix: Use correct max depth for Binance (and its child classes).Bugfix: Fix test data generation, fix Binance test cases, clean up and fix issues in various code samples in example/Feature: BinanceUS rest mixinUpdate: add feed/exchange cleanup to integration testsBugfix: Last message received not being correctly set on websocket connection, causing multiple restarts when an exchange encounters a timeoutBugfix: Binance Futures not correctly formatting the side on liquidationsBugfix: Interval from candle_sync was not being passed correctly to async candle interface in REST mixins.Update: Cleanup Coinbase candle REST interface, use standard string intervalFeature: Add balances to BybitBugfix: Kraken valid depths incorrectFeature: Add support for gracefully stopping Redis backends and writing queued messageBugfix: OKEx incorrect creating multiple connectionsBreaking Change: Data types for majority of callbacks have changed to Objects (previously was a dict)Update: Remove redundant example codeBreaking Change: OrderInfo now an objectBugfix: NBBO updated to use new orderbookBreaking Change: Balance callback changed to return objectBreaking Change: L1_Book callback returns objectUpdate: Subscribe to 200 levels per side for BybitFeature: Candles support added to Binance RESTBreaking Change: Candle REST methods return Candle objectFeature: data objects now hashable and comparable (equal only)Breaking Changes: USER_FILLS renamed FILLS, FILLS not use data objects for callbacksFeature: Add support for candles in FTX RESTFeature: Add support for candles in Bitstamp RESTFeature: Add support for candles in Upbit REST1.9.3 (2021-08-05)Feature: Add support for private channel USER_DATA, public channel LAST_PRICE on PhemexFeature: Add support for private channels FILLS, ORDER_INFO, BALANCES on DeribitFeature: Add support for public channel L1_BOOK on DeribitFeature: Add support for private channels FILLS and ORDER_INFO on BybitBugfix: Fix demo.pyFeature: Allow user to specify a delay when starting an exchange connection (useful for avoiding 429s when creating a large number of feeds)Update: Support Okex v5Breaking Change: Update symbol standardization. Now uses standard names across all exchanges for futures, swaps, and options.Feature: Allow user to specify depth_interval for Binance L2_BOOK.Bugfix: Use order id in FTX fill channel callbackFeature: Add ability to use the Symbols class to identify all exchanges that support a given instrumentFeature: Allow user to specify 'http_proxy' in feeds.Feature: Add support for 'concurrent_http' requests in Binance feeds.Bugfix: funding and open interest data not being collectedBreaking Change: Rework how REST endpoints are integrated into exchange classes. Rest module has been removed. REST methods are part of exchanges classes.Feature: Add support for funding data in BybitUpdate: Correct and update sections of the documentation.Feature: Add support for open_interest_interval in Binance Futures.Bugfix: Fix subaccounts impl in FTX1.9.2 (2021-07-14)Bugfix: add config kwarg to add_nbbo methodUpdate: changed KuCoin authentication to match new signing methodBugfix: #518 - fix aggregator example codeUpdate: Support Bittrex V3Feature: Add support for candles on BittrexFeature: Add support to authenticate private channels (e.g. FILLS) on FTXFeature: Support private rest api commands for FTXUpdate: Improve impl for FTX rest apiBugfix: #528 - Fix standardisation of Deribit's symbols when passed to callbacksFeature: Add support for private "orders" channel on FTXFeature: Add support for subaccounts in feeds and REST API for FTXBugfix: Fix FTX rest api return valueExchange: New exchange - dYdXBugfix: Issue #531 - Gemini symbol generation included closed symbolsFeature: Allow user to override the score used in Redis ZSETsUpdate: Get information about size increment from FTX symbol dataBugfix: Fix trades write for Arctic backendFeature: new exchange: Bequant. Supports ticker, L2 book, trades, candles, plus authenticated channels: order info, account transactions and account balancesUpdate: BitMax renamed AscendEXBugfix: Feed level timeout and timeout interval not being set properlyExchange: Phemex exchange supportFeatures: added support for candles, order info, account transactions and account balances to HitBTC & Bitcoin.com, plus authentication where required to access these channelsUpdate: previous HitBTC & Bitcoin.com websocket endpoints deprecated. Now using separate Market, Trading and Account endpointsBugfix: max_depth on Binance and Kraken was not properly used when querying the snapshotBugfix: Handle 429s in HTTP connections (by waiting and retrying).1.9.1 (2021-06-10)Feature: add Bithumb exchange - l2 book and tradesBugfix: Fix inverted Poloniex symbolsFeature: simplify and cleanup parts of PoloniexFeature: addsymbolsclass method to all exchanges to get list of supported trading pairsFeature: Clean up internal class attributes in Feed classFeature: Add graceful stop and shutdown methods for FeedsFeature: Add ledger endpoint to Kraken Rest module, add ability to optionally filter by symbol, or all symbols, for historical tradesDocs: Update documentation regarding adding a new exchange to custom_cryptofeedBugfix: Reset delay after connection is successfulFeature: yapic.json parses strings to datetimes automatically, no longer need to rely on Pandas for datetime parsingBugfix: #491 - dictionary resized during iteration in ByBitBugfix: #494 - added status argument to liquidations callbackBugfix: #399 - book delta issue with Kucoin and GateioFeature: Binance Delivery candle supportFeature: Binance US candle supportFeature: Kraken Candle supportUpdate: Remove deprecated channel mapping from Kraken, use channel name from message insteadBugfix: change Kraken Futures to use the standard symbol to be consistent with the rest of the libraryUpdate: use Kucoin v3 endpoint for orderbook snapshot (v2 deprecated).Update: Poloniex ticker message format update1.9.0 (2021-04-25)Bugfix: Fix Binance subscriptions when subscribing to more than one candleFeature: Remove support for Influx versions prior to 2.0Feature: Add stop method to HTTP Backends to gracefully drain queue and write pending data on shutdownFeature: Revamp InfluxDB code. Drop support for storing floating point as str, store book data as json blobBugfix: Remove unused get_instrument calls in Deribit and Kraken FuturesFeature: Revamp symbol generation and exchange info for Deribit and Kraken FuturesBugfix: Fix issue using AsyncFile callback to store raw dataTesting: Add exchange tests for Deribit and BinanceBugfix: Fix symbol issue in Bitmex when initializing the orderbookBugfix: Fix various issues with FTX, OKCOIN/OKX and Huobi symbol generationTesting: Overhaul exchange tests, all exchanges are now tested with real data. Fixed various bugs as a result of this testing. Revamped AsyncFileCallback.
Added new tool to generate test data for testing.Bugfix: Improve connection cleanup in AsyncConnection objectFeature: Add support for user defined exception handling in FeedHandlerBugfix: Fix redis backends that can't handle NoneBugfix: Connection exceptions being ignored in FeedhandlerBugfix: Binance address generation correctionBugfix: OKX symbol generation incorrect + validate symbols used for channels that dont support all typesBreaking Change: Large rewrite of Feedhandler, Connection, and Feed. Many timeout related options moved from feedhandler to Feed. Symbol specific code
moved to exchange class. Rewrite of raw data collection.Feature: Candle support for HuobiFeature: Allow user to specify Postgres port in Postgres backendsBugfix: Report base volume, not quote volume in Huobi candlesFeature: Support for the KuCoin exchange1.8.2 (2020-04-02)Update to use alpha release of aioredis 2.0. Allows building of wheels again1.8.1 (2020-04-01)Bugfix: Add manifest file for source dist1.8.0 (2020-04-01)Bugfix: Init uvloop earlier so backends that use loop will not failDocs: Remove FAQ, added performance doc sectionBugfix: #404 - Use AsyncConnection object for Binance OIFeature: Rework how raw data is stored (when enabled). REST data can now be capturedFeature: New feedhandler method,add_feed_runningallows user to add feed to running instance of a feedhandlerFeature: create_db defaults to False on InfluxDB backendsFeature: Normalize Bitmex SymbolsUpdate: Remove extraneous methods in feed objects used to query symbol informationFeature: Use realtime ticker for BinanceBugfix: Bitmex symbols not being normalized correctlyBugfix: Fix GCP PubSub backendBugfix: Fix historical data REST api for BitmexFeature: Use separate tasks (fed by async queue) for backend writing. Redis now uses this methodBugfix: Allow user specified max depths on KrakenFeature: Add backend queue support to ZMQ backendFeature: Add backend queue support to Socket backendsFeature: Add VictoriaMetrics support via backendFeature: Add backend queue support to influx and elasticFeature: Candle supportBugfix: Ignore untradeable symbols in Binance symbol generationFeature: Add backend support for queues in Postgres. Rework postgres backend and supply example SQL file to create tables for demoBugfix: Fix ByBit symbol generationFeature: Authenticated channel support for OKX/OKCOINUpdate: Poloniex changed signaure of ticker dataFeature: Candles for Binance FuturesFeature: Premium Index Candle support for Binance FuturesFeature: Update Gateio to use new v4 websocket api. Adds support for candlesBugfix: Fix open interest on OKExBugfix: OKEx was duplicating subscriptionsBreaking Change: Core callbacks (trade, candle, books, ticker, open interest, funding, liquidations, index) now use custom objects1.7.0 (2021-02-15)Feature: Use UVLoop if installed (not available on windows)Bugfix: Allow exchanges to customize their retry delays on errorFeature: New demo code showing user loop managementFeature: Handle more signals for graceful shutdownBugfix: BinanceFutures message format changeFeature: Missing sequence number on Coinbase will not reset all data streams, just the affected pairFeature: Use timestamp from exchange for L2 book data from CoinbaseBugfix: Blockchain exchange had incorrect timestamps, and incorrect log linesBugfix: Wrong datatype in BackendFuturesIndexCallbackBugfix: Fix bad postgres callback for open_interest and futures_indexFeature: Signal handler installation now optional, can be done separately. This will allow the feedhandler to be run from child threads/loopsBugfix: Fix binance delivery book ticker (message format change)Breaking change: Feed objectconfigrenamedsubscriptionFeature: Configuration passed from feedhandler to exchangesBreaking change: Most use ofpairandpairschanged tosymbolandsymbolsto be more consistent with actual usage. pairs.py renamed to symbols.pyFeature: Allow configuring the API KEY ID from Config or from environment variableBugfix: Collisions in normalized CoinGecko symbols (this adds about 700 new symbols)Feature: Add candles function to coinbaseFeature: Explain when custom_cryptofeed crashes during pairs retrievalBugfix: BINANCE_DELIVERY Ticker use msg_type='bookTicker' as for the other BINANCE marketsFeature: Support Bitmex authentication using personal API key and secretFeature: Print the origin of the configuration (filename, dict) for better developer experienceBugfix: Add guard against non-supported asyncio add_signal_handler() on windows platformsFeature: Simplify source code by standardization iterations over channels and symbolsBugfix: Remove remaining character "*" in book_test.pyBugfix: Fix return type of the function book_flatten()Feature: Shutdown multiple backends asynchronously, and close the event loop properlyBugfix: Repair the Bitfinex FUNDINGFeature: Speedup the handling of Bitfinex messages by reducing intermediate mappingsFeature: Support OKEx optionsBugfix: Cancel the pending tasks to gracefully/properly close the ASyncIO loopFeature: Support for authenticated websocket data channels1.6.2 (2020-12-25)Feature: Support for Coingecko aggregated data per coin, to be used with a new data channel 'profile'Feature: Support for Whale Alert on-chain transaction data per coin, to be used with a new data channel 'transactions'Bugfix: Reset delay and retry for rest feedFeature: Add GCP Pub/Sub backendBugfix: Fix aggregated callbacks (Renko and OHLCV) when used with exchanges that support order typesBugfix: Fix broken example/demo codeFeature: New data channel -futures_index- demonstrated in ByBitFeature: Add stop callback when exiting loop, add stop method placeholder for base callbacksBugfix: Fix NBBO callbackFeature: Orderbook sequence number validation for HitBTCFeature: Kraken orderbook checksum support in KrakenFeature: KrakenFutures sequence number check addedFeature: Add optional caching to postgres backendFeature: New Exchange - Binance DeliveryFeature: Liquidation for OKXBugfix: Adjust ping interval on websocket connection, some exchanges require pings more frequentlyFeature: Checksum validation for orderbooks on OKX and OKCoinFeature: Use rotating log handlerBugfix: Later versions of aiokafka break kafka backendBugfix: Huobi sends empty book updates for delisted pairsBugfix: Harden channel map usage in KrakenFeature: Config file supportBugfix: Subscribing to all BitMEX symbols gives 400 error - message too longBugfix: Cleanup of code - fixed a few examples and resolved all outstanding flake8 issuesBugfix: Fix Bitfinex pair normalizationFeature: Refactor connection handling. New connection design allows feeds to open multiple connectionsFeature: Update BitMax to use the new BitMax Pro API - includes sequence number verification on booksFeature: Bybit - support for USDT perpetual data channelsFeature: Can now configure more than 25 Bitfinex pair/channel combinationsFeature: Support more than 200 pair/stream combinations on Binance from a single FeedFeature: Support for the bitFlyer exchangeFeature: Update Kraken to work with very large numbers of trading pairs1.6.1 (2020-11-12)Feature: New kwarg for exchange feed -snapshot_interval- used to control number of snapshot updates sent to clientFeature: Support for rabbitmq message routingFeature: Support for raw file playback. Will be useful for testing features and building out new test suites for custom_cryptofeed.Feature: Arctic library quota can be configured, new default is unlimitedFeature: New exchange: ProbitBugfix: Correctly store receipt timestamp in mongo backendBugfix: FTX - set a funding rate requests limit constant (10 requests per second, 60 seconds pause between loops)Bugfix: Open Interest data on FTX erroneously had timestamps set to NoneUpdate: Binance Jersey shutdown - feed removedBugfix: Fixed open interest channel for Binance Delivery1.6.0 (2020-09-28)Feature: Validate FTX book checksums (optionally enabled)Bugfix: Subscribing only to open interest on Binance futures gave connection errorsFeature: Authentication for Influxdb 1.xFeature: Override logging defaults with environment variables (filename and log level)Bugfix: For Coinbase L3 books need to ignore/drop some change updates (per docs)Bugfix: Obey rate limits when using Coinbase REST API to get L3 book snapshotsBugfix: Ignore auction updates from GeminiFeature: Add order type (limit/market) for Kraken TradesFeature: Exchange specific information available via info classmethod - contains pairs, data channels and tick sizeFeature: Funding data supported on HuobiSwapBugfix: Fix broken mongo callbacks in backends1.5.1 (2020-08-26)Bugfix: #136 - Kraken Rate limitingFeature: Funding data on Binance FuturesBugfix: Support new Huobi tradeId field, old id field deprecatedBugfix: Unclear errors when unsupported data feeds usedBugfix: Handle order status messages more gracefully in CoinbaseBugfix: Fix Kraken pair mappingsFeature: New Exchange - Gate.ioFeature: Remove _SWAP, _FUTURE channel (and callback) types - determine correct type at subscribe time based on symbolDocs: Add documentation about callbacksFeature: Deribit provides sequence number for book updates - check them to ensure no messages lostBugfix: Fix timestamp on Binance Futures Open InterestBugfix: Update/standardize liquidation callbacksFeature: Update Upbit subscription methods based on updated docsBugfix: Ticker not working correctly on Binance FuturesFeature: Liquidations callbacks for backends1.5.0 (2020-07-31)Feature: New Exchange - FTX USFeature: Add funding data to rest libraryBugfix: DSX updated their API, websocket no longer supported. Removing DSXFeature: Websocket client now uses unbounded message queueFeature: Support for HuobiDM next quarter contractsBugfix: Fix datetime fields in elasticsearchFeature: BinanceFutures: support ticker, open interest and Liquidation, FTX: support open interest and liquidations, Deribit: liquidations supportBugfix: Fix receipt timestamps in Postgres backendBugfix: Huobi Swap Init1.4.1 (2020-05-22)Feature: Support for disabling timeouts on feedsBugfix: #224 Ignore newly added trading pairs in Poloniex while runningFeature: New exchange, DSXBugfix: Bybit updated their API, websocket subscription to L2 book data needed to be updatedBugfix: Deribit subscription condensed into a single message to avoid issues with rate limitBugfix: Funding interval for bitmex not converted to integerBugfix: HuobiSwap missing from feedhandlerFeature: Optional flag on Feed to enable check for crossed booksFeature: Blockchain Exchange1.3.1 (2020-03-17)Feature: Add missing update detection to orderbooks in BinanceFeature: REST support for FTXFeature: Added new field, receipt timestamp, to all callbacks. This contains the time the message was received by custom_cryptofeed.Feature: Upbit Exchange Support1.3.0 (2020-02-11)Bugfix: Enabling multiple symbols on Bitmex with deltas and max depth configured could cause crashes.Bugfix: Default open interest callback missingChange: Mongo backend stores book data in BSONFeature: Open Interest callbacks added to all backendsChange: Instrument removed in favor of open interestBugfix: Huobi feedhandlers not properly setting forced indicator for book updates, breaking deltasBugfix: Some Kraken futures funding fields not always populatedFeature: Open interest updates for Kraken futuresFeature: Open interest updates for DeribitBugfix: FTX ticker can have Nones for bid/askFeature: InfluxDB 2.0 supportBugfix: Deribit funding only available on perpetualsFeature: Enable deltas (with out max depth) on exchanges that do not support them1.2.0 (2020-01-18)Feature: New exchange: Binance FuturesFeature: New Exchange: Binance JerseyFeature: Funding data on Kraken FuturesFeature: User defined pair separator (default still -)Feature: Postgres backendFeature: Deribit FundingBugfix: Deribit subscriptions using config subscribed to symbols incorrectlyBugfix: Some RabbitMQ messages were missing symbol and exchange dataFeature: Open interest data for OKX swaps1.1.0 (2019-11-14)Feature: User enabled logging of exchange messages on errorRefactor: Overhaul of backends - new base classes and simplified codeBugfix: Handle i messages from poloniex more correctlyBugfix: Report bittrex errors correctlyFeature: New exchange: Bitcoin.comFeature: New exchange: BinanceUSFeature: New exchange: BitmaxFeature: Ability to store raw messages from exchanges1.0.1 (2019-09-30)Feature: Backfill Bitmex historical trade data from S3 BucketFeature: RabbitMQ backendFeature: Custom Depth and deltas for all L2 book updatesFeature: Support new 100ms book diff channel on BinanceFeature: Bittrex exchange supportFeature: Ticker support in Redis and Kafka BackendsFeature: Ticker callbacks require/contain timestampFeature: Renko AggregationBugfix: Max Depth without deltas should only send updates when book changesBugfix: Update count and previous book now associated with pair1.0.0 (2019-08-18)Bugfix #113: Fix remaining exchanges who are not reporting timestamps correctlyFeature: Generated timestamps now based on message receipt by feedhandlerFeature: Multi-callback supportFeature: Rework ZMQ using pub/sub with topicsFeature: FTX ExchangeFeature: Gemini subscriptions now work like all other exchangesFeature: Use unique id for each feed (as opposed to feed id/name)Bugfix: fix Poloniex historical trade timestampsBugfix: Bitmex L2 channel incorrectly classifiedFeature: Kraken FuturesFeature: Redis backend supports UDSFeature: Binance full book (L2) with deltasFeature: Allow user to start event loop themselves (potentially scheduling other tasks before/after).0.25.0 (2019-07-06)Feature: Rest Endpoints for Historical Deribit dataFeature: Specify numeric datatype for InfluxDBBugfix: Greatly improve performance of book writes for InfluxDBFeature: Bybit exchange supportBugfix: Deribit now returning floats in decimal.DecimalFeature: Elastic Search backend0.24.0 (2019-06-19)Bugfix: Book Delta Conversion issue in backendsBugfix: Tweak BitMEX rest API to handle more errors more gracefullyFeature: Deribit Exchange supportFeature: Instrument channelBugfix: support Kraken websocket API changesBugfix: correct USDT symbol mappings for BitfinexBugfix: Fixed mongo book backendFeature: Book delta support for mongo, sockets, ZMQ0.23.0 (2019-06-03)Feature: Book delta support for InfluxDBFeature: Swaps on OkEX0.22.2 (2019-05-23)Bugfix: Fix tagging issue in InfluxDBBugfix: Fix book updates in InfluxDBFeature: Book delta support in Redis backendsFeature: Book delta support in Kafka backend0.22.1 (2019-05-19)Feature: Cleanup callback codeFeature: Poloniex subscription now behaves like other exchangesFeature: Kafka Backend0.22.0 (2019-05-04)Bugfix: Timestamp normalization for backends were losing subsecond fidelityFeature: All exchanges report timestamps in floating point unix timeBugfix: Implement change in OkEx's trading pair endpoint for pair generation0.21.1 (2019-04-28)Feature: Config support for Coinbene, Binance, EXX, BitMEX, Bitfinex, Bitstamp, HitBTCFeature: Complete clean up of public REST endpointsFeature: Improved book delta exampleFeature: Bitstamp Websocket V2 - L3 books now supportedBugfix: Incorrect book building in Kraken0.21.0 (2019-04-07)Bugfix: Coinbase L3 Book would get in cycle of reconnecting due to missing sequence numbersFeature: Kraken L2 Book DeltasFeature: Book deltas streamlined and retain orderingFeature: OKCoin exchange supportFeature: OKEx exchange supportFeature: Coinbene exchange supportFeature: Support Huobi Global and Huobi USA0.20.2 (2019-03-19)Bugfix: Kraken REST API using wrong symbol for tradesFeature: Complete work on standardizing Bitfinex rest APIBugfix: Allow index symbols on Bitmex0.20.1 (2019-02-16)Feature: Trades sides are now labeled as Buy / Sell instead of Bid / Ask.Feature: Support for the Huobi exchangeBugfix: Change how exchange pairs are mapped for REST module - only map exchanges that are usedBugfix #67: Ensure all trades report the taker's side0.20.0 (2019-02-04)Feature #57: Write updates directly to MongoDB via new backend supportFeature #56: Experimental support for fine grained configuration per exchangeFeature #58: Support Kraken websocket APIFeature: Only generate trading pair conversions for configured exchangesFeature: Historical trade data on REST API for Kraken0.19.2 (2019-01-21)Feature #55: OHLCV aggregation method in backends plus support for user defined aggregatorsFeature: EXX exchange support0.19.1 (2019-01-11)Bugfix: Poloniex logging had bug that prevented reconnect on missing sequence number0.19.0 (2019-01-10)Feature #50: Support multiple streams per websocket connection on BinanceBugfix #51: Fix pairs on streams in Binance0.18.0 (2018-12-15)Feature: InfluxDB support via backendFeature: Aggregation backend wrappersBugfix: BookDelta callback no longer needs to be an instance of BookUpdateCallbackBugfix: REST module was creating duplicate log handlersBugfix: Bitfinex REST now properly handles cases when there are more than 1000 updates for a single tick0.17.4 (2018-11-17)README change for long description rendering issue0.17.3 (2018-11-17)Feature #41: Rework trading pairs to generate them dynamically (as opposed to hard coded)Feature: When book depth configured Redis, ZMQ and UDP backends only report book changes when changed occurred in
depth windowFeature: TCP socket backend supportFeature: UDS backend support0.17.2 (2018-11-03)Bugfix #45: Bitstamp prices and sizes in L2 book are string, not decimal.DecimalFeature: Binance support0.17.1 (2018-10-19)Bugfix #43: Coinbase L2 book used "0" rather than 0 for comparisons against decimal.DecimalFeature: REST feed market data supported via normal subscription methodsFeature: Kraken supportBugfix: Bitfinex book timestamps match expected Bitfinex timestamps (in ms)0.17.0 (2018-10-13)Feature: Timestamps for orderbooks and book deltasFeature #40: NBBO now uses best bid/ask from L2 booksFeature #28: GDAX now renamed Coinbase and uses Coinbase endpointsFeature: ZeroMQ backend. Write updates directly to ZMQ connectionFeature: UDP Socket backend. Write updates directly to UDP socket0.16.0 (2018-10-4)Feature: L2 books are now all price aggregated amounts, L3 books are price aggregated ordersBook deltas supported on all feedsBugfix: Fix NBBO feed0.15.0 (2018-09-29)Feature: GDAX/Coinbase rest support - trades, order status, etcFeature: Arctic backend, supports writing to arctic directly on trade/funding updatesBugfix: #36 Update poloniex to use new trading pairs and handle sequence numbersBugfix: Improve Bitfinex orderbooks and handle sequence numbersBugfix: GDAX and Bitmex orderbook and logging improvements0.14.1 (2018-09-14)Added some docstringsFeature: Add exchanges by name to feedhandler. Easier to instantiate a feedhandler from configLogging improvementsBugfix: non-gathered futures were suppressing exceptions when multiple feeds are configured. Changed to tasksRedis backend uses a connection pool0.14.0 (2018-09-04)Feature: support for writing order books directly to RedisFeature: ability to specify book depth for Redis updates0.13.3 (2018-08-31)Feature: normalize Bitfinex funding symbols0.13.2 (2018-08-31)Bugfix: fix symbol in Bitfinex rest0.13.1 (2018-08-31)Feature: access rest endpoints via getitem / []Bugfix: #31 - funding channel broke GeminiFeature: Book deltas for GDAXBugfix: Fix intervals on Bitmex (rest)0.13.0 (2018-08-22)Feature: Funding data from Bitmex on wsFeature: Funding historical data via restBugfix: Python 3.7 compatibilityFeature: Rest trade APIs are now generatorsFeature: funding data on Bitfinex - ws and rest0.12.0 (2018-08-20)Bugfix: Handle 429s in Bitmex (REST)Feature: Redis backend for trades to write updates directly to RedisBugfix: issue #27 - Bitmex trades missing timestamps0.11.1 (2018-08-18)Bitfinex and Bitmex historical trade data via RESTBugfix: interval incorrect for rest time rangesBugfix: lowercase attrs in Rest interface0.11.0 (2018-08-05)Feature: Support for delta updates for order booksREST API work started0.10.2Bugfix: Clear data structures on reconnect in bitmexFeature: Support reconnecting on more connection errorsFeature: Timestamp support on trade feedsFeature: Connection watcher will terminate and re-open idle connections0.10.1 (2018-5-11)Feature: Reconnect when a connection is lostBugfix #22: Check for additional connection failuresFeature #4: Trade ID supportFeature: Account for new Gemini message type0.10.0 (2018-03-18)Feature: Bitmex0.9.2 (2018-03-13)Bugfix #10: Change from float to decimal.Decimal in GDAXFeature #5: use sorted dictionaries for order booksFeature #17: logging supportBugfix: Gemini order books now workBugfix: All json floats parsed to DecimalBugfix: Fix Bitstamp pair parsingFeature: Major clean up of channel, exchange, and trading pair names0.9.1 (2018-01-27)Bugfix #4: produce ticker from trades channel on GDAXFeature: Bitstamp feed0.8.0 (2018-01-07)Feature: HitBTC feedFeature: Poloniex Orderbook support0.6.0 (2018-01-02)Feature: Gemini Feed0.5.0 (2018-01-02)Initial release: GDAX, Poloniex, Bitfinex SupportFeature: NBBO support