Package: cubyc
Examples|Install|FAQ|Quickstart|License|ContributingThe data science framework for decision-making researchMake the best decisions, consistently. Research decision-making processes with your data and deploy them with confidence.🐍 Declare your processes in PythonDefine and combine asynchronous schedules and events, and parameterize your processes with Pydantic fields.See exampleimportrandomfromcubycimportProcess,schedulesclassMyProcess(Process):counter:int=0defsetup(self):(self.bewteen(hours=(8,12))&self.every(minutes=5)).do(self.morning_function)self.when(lambda:notself.counter%10).do(self.random_function)@schedules.every(hours=1)defhourly_function(self):print("I run every hour")self.counter+=random.randint(1,100)defmorning_function(self):print("I run every 5 minutes between 8AM and noon")self.counter+=1defrandom_function(self):print("I run when the counter is a multiple of 10")⏳ Backtest locally, deploy to productionBacktest your decision-making processes on historical data and deploy them to production in a single line of code.See examplefromdatetimeimporttimedeltaprocess=MyProcess(counter=10)# backtests the 2023 yearprocess.run(start="2023-1-1",end="2023-12-31")# backtests from 2023 to today, and runs live for 10 daysprocess.run(start="2023-1-1",end=timedelta(days=10))# runs live for 100 daysprocess.run(end=timedelta(days=100))🚀 Integrate any data sourceAccess your data and utilize it across processes with modular data feeds.See examplefromdatetimeimporttimedeltafromcubycimportProcess,DataFeedclassMyDataFeed(DataFeed):defname(self):return'my_data_feed'defload_from_source(self,start_datetime,end_datetime):# query and return data from one of your data sources...classMyProcess(Process):datafeed:DataFeed=MyDataFeed()defsetup(self):self.datafeed.last(values=10)# returns the last 10 valuesself.datafeed.last(timedelta(days=5))# returns the last 5 days of dataself.datafeed.current()# returns the current value️🌊  Monitor and visualize your processesTrack experiments, visualize logs, and evaluate your decision-making processes with Cubyc’s Push platform.See exampleComing soon!Stay in touchfor updates.Install CubycSimple installation from PyPIpipinstallcubycOther installation optionsFrom sourceClone the repository, navigate to the root directory, and runpythonsetup.pyinstallFAQHow is Cubyc different from workflow orchestration tools?Workflow orchestration automates repetitive tasks or processes, specifically handling the movement of information or data between systems. 
Conversely, Cubyc serves as a decision support system (DSS) to automate decision-making and optimize or improve a specific outcome.Algorithmic DSSWorkflow OrchestratorsPurposeAutomate decisions and improve or optimize outcomesAutomate repetitive tasks or processesUse CasesAlgorithmic trading, recommendation systems, Ad campaigns, pricing enginesETL, order fulfillment, customer onboarding, DevOps AutomationLibrariesCubycAirflow, Luigi, Prefect, DagsterFrequencyHigh-frequency, real-time decision-makingLow-frequency, batch processingHow do backtests improve decision-making?Evaluation metrics such as accuracy, precision, and recall offer insights into model performance but lack actionable information
on decision-making effectiveness. For example, a model with 90% test accuracy may not necessarily improve revenue.
To assess impact, actionable metrics like ROI, revenue, and conversions rate are crucial. 
Backtests offer actionable insights into the potential impacts of different decisions through simulations of past events.
For instance, they can help understand:
- A portfolio's sharpe ratio over the past 10 years with monthly vs quarterly rebalancing
- An ad campaign's annual ROI with different budget allocations 
- A marketplace's 2020 revenue with alternative pricing processes
- A service's historical fraud rate with adjustments to the model's thresholdQuickstartCubyc is divided into thecoreanddatamodules, and theresearchplatform.🔬 CoreThecoremodule contains the classes and functions that power Cubyc's Process with scheduling, backtesting,
and logging.Learn moreProcessThecubyc.core.Processclass is Cubyc's main class, providing a interface to define your schedules and events.fromcubyc.coreimportProcess,schedulesclassMyProcess(Process):hyperparameter:intdef__init__(self,variable:int,*args,**kwargs):super().__init__(*args,**kwargs)self.variable=variabledefsetup(self):self.variable=self.hyperparameter@schedules.every(hours=1)defupdate(self):self.variable+=1self.log(f'Variable:{self.variable}')Process=MyProcess(variable=0,hyperparameter=7)SetupThesetup()function gets called before your Process is run live or backtested.
Be sure to define your Process's initial state here!fromcubycimportProcessclassMyProcess(Process):defsetup(self):# define your initial state hereself.state_boolean=Falseself.state_counter=0Versioning and hyperparametersIn addition to passing arguments to your Process's__init__constructor, you can also
define attribute hyperparameters with Pydantic's field types.fromcubycimportProcessclassMyProcess(Process):# define your hyperparameters using SQLAlchemy's ORM-style hererepeats:intname:strdef__init__(self,*args,**kwargs):super().__init__(*args,**kwargs)for_inrange(self.repeats):self.log(f'Hello,{self.name}!')my_process=MyProcess(hyperparameter_1=3,hyperparameter_2='Jens')If you're using Cubyc's Experiment as a Service (EaaS) platform, your run hyperparameters will be automatically
in the cloud.NoteBe sure to add type hints to your hyperparameters.Structured and Unstructured LoggingLogging messages or variables in Cubyc is easy! Simply callself.log()the message or variable(s) you want to log.Unstructured logging: Pass a string toself.log()to log a message to the console.Structured logging: Pass a dictionary toself.log()to record the variable(s) for later analysis.fromcubycimportProcessclassMyProcess(Process):defsetup(self):self.log('Hello world!')self.log({'variable':1,'another_variable':2})Will log the following to console:[07/03/23 10:30:00] INFO     Hello World!And generate the following record in the Process's .csv logfile:timestampkeyvalue2023-07-03 10:30:00variable12023-07-03 10:30:00another_variable2You can then query the Process's structured .csv logfile with thelogs()function.run=my_process.do(...)run.logs('variable')The above code would return apd.DataFramecontaining all the Process's structured logs for the keyvariable.SchedulingCubyc'score.scheduling.schedulersmodule provides a powerful scheduling functionality for your Process.fromcubyc.coreimportschedulesfromcubycimportProcessclassMyProcess(Process):defsetup(self):self.every(hours=1).do(self.say_hello)defsay_hello_world(self):print('Hello, world!')@schedules.market_open(market='NYSE')defupdate(self):print('Updating...')self.after(hours=1).do(lambda:print('One hour later...'))Scheduling PrimitivesCubyc defines six scheduling primitives that you can use to create your Process:onruns a function on a specific date and time.everyruns a function everynyears, months, days, hours, minutes or seconds.betweenruns a function between two dates.afterruns a function after a specific date.whenruns a function when a specific event occurs.onceruns a function when a specific event occurs, but only once.Simple SchedulesThe execution times of simple schedules (on,every,between,after) is deterministic and
can be calculated in advance. To create a simple schedule, you can callschedule=self.<SCHEDULE>(...)Then, run a function with said schedule by calling therunmethod with the function as a parameter:schedule.do(<FUNCTION>)For example, to create a schedule that runs anupdatefunctionevery4 hours, you can write:self.every(hours=4).do(self.update)Similarly, to schedule the functiondo_something_else15 daysafterrunning thedo_somethingfunction:defdo_something(self):self.after(days=15).do(self.do_something_else)Decorated Simple SchedulesYou can also create simple schedules by decorating functions with the desired simple schedule.
For example, to schedule your Process'shourly_updatefunction to run every hour, andweekday_updateto run between weekdays, we can decorate them as:fromcubycimportProcess,schedulesclassMyProcess(Process):@schedules.every(hours=1)defhourly_update(self):pass@schedules.between(days_of_week='mon-fri')defdaily_update(self):passConditional SchedulesSometimes, we want to run a function when a specific random event occurs. Conditional schedulers (when,once)
allow us to do this by taking a function that returns a boolean value as a parameter, and running the function
when the function returnsTrue.For example, to schedule theupdate()when the functioncondition()returnsTrue,
we write the following code:importnumpyasnpfromcubycimportProcessclassMyProcess(Process):defsetup(self):self.counter=0self.when(self.condition).do(self.update)defcondition(self):# returns True when a specific condition is metreturnnp.random.rand()<0.1defupdate(self):# update logic hereself.counter+=1If we instead wanted to run the functionupdateonlyonceafterconditionreturnsTrue, we would instead schedule it with:defsetup(self):self.once(self.condition).do(self.update)Compound SchedulesSchedules can be combined to create compound schedules. To do this, you can use the&and|operators to
combine schedulers with a logical AND or OR respectively.For example, if we want to run theupdatefunction every 5 minutes and
between 10 AM and 4:00 PM, we would define our setup function as:defsetup(self):(self.every(minutes=5)&self.between(hours=(10,16))).do(self.update)Alternatively, if we wanted to run the same function between 10 AM and 12:00 PM or between 1:00 PM and 4:00 PM:defsetup(self):(self.between(hours=(10,12)|self.between(hours=(13,16)))).do(self.update)Now let's put it all together! Suppose we want to run the functionupdateevery 5 minutes,
between 10 AM and 4:00 PM, and only on the 3rd of the month:defsetup(self):(self.every(minutes=5)&self.between(hours=(10,16))&self.on(day=3)).do(self.update)NoteYou can use lambda functions to create schedules with parameters.📈 DataThedatamodule at Cubyc enables you to easily load, cache, and access data from any source.Learn moreData FeedCubyc'sDataFeedclass is a pluggable interface for loading data from any data source.Creating a Data FeedData feeds automate the process of loading, caching data, as well as retrieving data with the correct frequency and format.
They can be easily created by subclassing theDataFeedclass and implementing only four methods:[required]name(): Returns the name of the data feed.[required]load(start_datetime, end_datetime): Loads data from your data source between the specified dates.[optional]schema(): Returns a dictionary defining the data feed's schema for each column.[optional]metadata(): Returns a dictionary containing the data feed's metadata for each column.The following snippet defines  a custom data feed that loads data from
the Federal Reserve'sFRED API:importnumpyasnpfromdatetimeimportdatetimeimportpandasaspdfromcubyc.dataimportDataFeedclassFREDDataFeed(DataFeed):def__init__(self,api_key):super().__init__()self.api_key=api_keydefname(self):return'FRED'defschema(self)->dict:return{'open':np.dtype(str),'close':np.dtype('float64')}defmetadata(self)->dict:return{'series_id':'FRED series ID from https://fred.stlouisfed.org/categories','value':'Numerical value of the series.'}defload_from_source(self,start_datetime:datetime,end_datetime:datetime,**kwargs):# Load data from FRED's REST API herereturnpd.DataFrame(...)Under the hood, Cubyc automatically check that the data you are loading from your data source matches the specified schema.
You can also call<DATAFEED-OBJECT>.info()to get a summary of the datafeed's schema and metadata.fred_datafeed=FREDDataFeed('FRED-API-KEY-ID')fred_datafeed.info()This would return:dtypemetadataseries_id<U0FRED series ID fromhttps://fred.stlouisfed.org/categoriesvaluefloat64Numerical value of the series.To add a data feed to your Process, simply add your data feed as a Pydantic field to your Process's class.classMyProcess(Process):FRED_datafeed:DataFeed=FREDDataFeed('FRED-API-KEY-ID')defupdate(self):data=self.FRED_datafeed.last(...)...Accessing DataDataFeeds have four important functions you need to be aware of:next: Returns the next data point(s) in the data feed, given the current time.last: Returns the last data point(s) in the data feed, given the current time.current: Returns the current data point(s) in the data feed if available.between: Returns all data points between two dates.For example, to retrieve the last 10 reviews from a Yelp data feed:self.yelp_datafeed.last(values=10,columns=['reviews'],company='Chipotle')Alternatively, if we wanted to retrieve the last 24 hours of movie reviews from a IMDb data feed:self.imdb_datafeed.last(values=datetime.timedelta(hours=24),columns=['reviews'],movie='The Dark Knight')To retrieve Apple's (AAPL) volume from Yahoo Finance datafeed between 2020-01-01 and 2023-01-11:self.yf_datafeed.between(start_date=datetime.datetime(2020,1,1),end_date=datetime.datetime(2023,1,11),columns='vwap')And finally, to retrieve the current price of a data feed:self.yelp_datafeed.current(columns='price',company='Chipotle')NoteData retrieved by data feeds are cached by default to a 'cubyc.sqlite' database in the current directory.
To cache data to a different location, specify theurlparameter in the data feed's constructor.Data StreamsThe built-inDataStreamclass enables streaming of real-time data from WebSockets APIsCreating a Data StreamTo stream data from a WebSocket API, you can subclass the ```DataStream``` class and implement the `run` methodfromcubycimportDataStreamfrompolygonimportWebSocketClientclassPolygonDataStream(DataStream):def__init__(self,polygon_api_key:str):self.ws=WebSocketClient(api_key=polygon_api_key)defrun(self,func):self.ws.do(handle_msg=func)Streamingto connect to the WebSocket API with your custom data stream, simply use thelistendecorator:fromcubycimportProcess,schedulesclassMyProcess(Process):@schedules.listen(PolygonDataStream('POLYGON-API-KEY-ID'))defupdate(self,msg):# handle msg here...WarningData streams are only supported during live deployment.🎨 ResearchCubyc'sresearchcontrol panel provides a powerful interface to visualize and monitor your processes.Learn moreExperiment as a Service (EaaS)Cubyc's Experiment as a Service (EaaS) platform provides a powerful interface to visualize and monitor your processes.
It allows you to track and version experiments, visualize logs, and evaluate different hyperparameter configurations
of your decision-making processes.RunsIf you provided an API key, Cubyc automatically logs your process's runs to the cloud.
You can view all your runs in the cloud under theRunstab in the research control panel.LogsWhen logging structured data, Cubyc automatically logs your structured data to the cloud in a tabular format.
Logs can be viewed and queried in theLogstab in the research control panel.License & ContactCubyc is distributed under theGNU Lesser General Public License.
Contact us atgrowth@cubyc.comfor any questions.ContributingWe welcome contributions from the community! If you have any ideas, suggestions, or bug reports,
please open an issue or a pull request.