Package: scramjet-client-utils
Scramjet Transform Hub API Clients utilityThe package provides utilities for API clients. Each of the API clients make use out of it.Usage:importasynciofromclient.host_clientimportHostClientfromclient_utils.base_clientimportBaseClient# your middleware tokentoken=''# set the tokenBaseClient.setDefaultHeaders({'Authorization':f'Bearer{token}'})# middleware urlapi_base='https://api.scramjet.cloud/api/v1'# url = {middlewareURL}/space/{manager_id}/api/v1/sth/{host_id}/api/v1host=HostClient(f'{api_base}/space/org-aa5bu150-9o5c-489b-83e3-b1yf7e086f3h-manager/api/v1/sth-1/api/v1')# get a list of Sequencesres=asyncio.run(host.list_sequences())print(res)More reading⭐ Star us on GitHub — it motivates us a lot! 🚀In the link below you will find more information about our stream protocol and API usage.See the code documentation here:scramjetorg/transform-hub/docs/read-more/stream-and-api.mdScramjet Transform HubThis package is part ofScramjet Transform Hub.Scramjet Transform Hub is a deployment and execution platform. Once installed on a server, it will allow you to start your programs and keep them running on a remote machine. You will be able to start programs in the background or connect to them and see their output directly on your terminal. You will be able to pipe your local data to the program, as if it was running from your terminal. You can start your server in AWS, Google Cloud or Azure, start it on your local machine, install it on a Raspberry Pi or wherever else you'd like.Use casesThere's no limit what you can use it for. You want a stock checker? A chat bot? Maybe you'd like to automate your home? Retrieve sensor data? Maybe you have a lot of data and want to transfer and wrangle it? You have a database of cities and you'd like to enrich your data? You do machine learning and you want to train your set while the data is fetched in real time? Hey, you want to use it for something else and ask us if that's a good use? Ask usvia emailor hop on ourScramjet Discord!Host operations[ GET ]/api/v1/version- show the Host versionParametersNo parametersResponsesSuccessful operation code:200{"version":"0.12.2"}[ GET ]/api/v1/sequences- show all Sequences saved on the HostParametersNo parametersResponsesSuccessful operation code:200[{"instances":[],// a list of all running Instances of this Sequence"id":"eea8bc33-440f-4a17-8931-eb22a17d5d56",// Sequence ID"config":{"container":{"image":"scramjetorg/runner:0.12.2","maxMem":512,"exposePortsRange":[30000,32767],"hostIp":"0.0.0.0"},"name":"@scramjet/hello-alice-out","version":"0.12.2","engines":{"node":">=10"},"config":{},"sequencePath":"index",// a path to file with a main function"packageVolumeId":"eea8bc33-440f-4a17-8931-eb22a17d5d56"}},{"instances":["02381acf-cb16-4cff-aa9b-f22f04ada94f"],"id":"3ec02b93-4ca9-4d23-baab-048dab5ffda4","config":{"container":{"image":"scramjetorg/runner:0.12.2","maxMem":512,"exposePortsRange":[30000,32767],"hostIp":"0.0.0.0"},"name":"@scramjet/checksum-sequence","version":"0.12.2","engines":{},"config":{},"sequencePath":"index.js","packageVolumeId":"3ec02b93-4ca9-4d23-baab-048dab5ffda4"}}][ GET ]/api/v1/instances- show all Instances running on the HostParametersNo parametersResponsesSuccessful operation code:200[{"id":"02381acf-cb16-4cff-aa9b-f22f04ada94f",// Instance ID"sequence":"3ec02b93-4ca9-4d23-baab-048dab5ffda4"// Sequence ID},{"id":"ab0272d8-c9b0-43f7-9e7e-bcac9ec0f21f","sequence":"e4ca555c-ced1-4a13-b531-f43016eaf4ed"}][ GET ]/api/v1/load-check- monitor CPU, memory and disk usage metrics on the Host machineParametersNo parametersResponsesSuccessful operation code:200{"avgLoad":0.08,"currentLoad":5.190776257704936,"memFree":4634816512,"memUsed":8050364416,"fsSize":[{"fs":"/dev/sda1","type":"ext4","size":20838993920,"used":14939455488,"available":5882761216,"use":71.75,"mount":"/"},{"fs":"/dev/sda15","type":"vfat","size":109422592,"used":9621504,"available":99801088,"use":8.79,"mount":"/boot/efi"}]}[ GET ]/api/v1/log- monitor CPU, memory and disk usage metrics on the Host machineParametersNo parametersResponsesContent-type:application/octet-streamSuccessful operation code:2002021-11-19T16:04:47.094Zlog(object:Host)Hostmaincalled.2021-11-19T16:04:47.100Zinfo(object:SocketServer)Serveron:/tmp/scramjet-socket-server-path2021-11-19T16:04:47.104Zinfo(object:Host)APIlisteningon:127.0.0.1:80002021-11-19T16:05:08.228Zinfo(object:Host)Newsequenceincoming...2021-11-19T16:05:08.229Zlog(object:LifecycleDockerAdapterSequence)Dockersequenceadapterinit.2021-11-19T16:05:08.229Zlog(object:DockerodeDockerHelper)Checkingimagescramjetorg/pre-runner:0.12.22021-11-19T16:05:12.234Zinfo(object:LifecycleDockerAdapterSequence)Dockersequenceadapterdone.2021-11-19T16:05:12.246Zlog(object:LifecycleDockerAdapterSequence)Volumecreated.Id:c50fe4d3-89cc-4685-a82a-16cbc744733d2021-11-19T16:05:12.246Zlog(object:LifecycleDockerAdapterSequence)StartingPreRunner{image:'scramjetorg/pre-runner:0.12.2',maxMem:128}2021-11-19T16:05:13.536Zlog(object:DockerodeDockerHelper)Checkingimagescramjetorg/runner:0.12.22021-11-19T16:05:16.670Zinfo(object:SequenceStore)Newsequenceadded:c50fe4d3-89cc-4685-a82a-16cbc744733d2021-11-19T16:05:16.672Zinfo(object:Host)Sequenceidentified:{container:{image:'scramjetorg/runner:0.12.2',maxMem:512,exposePortsRange:[30000,32767],hostIp:'0.0.0.0'},name:'@scramjet/multi-outputs',version:'0.12.2',engines:{},config:{},sequencePath:'index.js',packageVolumeId:'c50fe4d3-89cc-4685-a82a-16cbc744733d'}2021-11-19T16:05:16.691Zdebug(object:Host)Requestdate:2021-11-19T16:05:08.239Z,method:POST,url:/api/v1/sequence,status:202Sequence operations[ POST ]/api/v1/sequence- add new sequenceParametersNameTypeDescriptionRequiredfilebinarycompressed package in tar.gz formatyesappConfigjsonadditional package.json config filenoResponsesAccepted operation code:202{"id":"2c3068e5-7c74-45bb-a017-1979c41fc6d0"// sequence id}[ GET ]/api/v1/sequences- show list of sequencesParametersNo parametersResponsesSuccessful operation code:200[{"instances":["742d2713-7ab6-4cde-82f3-a7beabdd4e98"// list of sequence instances],"id":"bdef63db-d3a0-45c8-85db-e94ebb96097f",// sequence id"config":{"container":{"image":"scramjetorg/runner:0.12.2","maxMem":512},"name":"@scramjet/transform-hub","version":"0.12.2","engines":{},"config":{},"sequencePath":"index.js","packageVolumeId":"bdef63db-d3a0-45c8-85db-e94ebb96097f"}}][ POST ]/api/v1/sequence/:id/start- start chosen sequenceParametersNameTypeDescriptionRequiredappConfigjsonadditional package.json config filenoargsjsonadditional arguments that instance should starts withnoResponsesSuccessful operation code:200{"id":"681c856e-dfa4-46a1-951d-47b27345552e"}[ DELETE ]/api/v1/sequence/:id- delete a sequence by idParametersNo parametersResponsesSuccessful operation code:200{"id":"2c3068e5-7c74-45bb-a017-1979c41fc6d0"}Conflict operation code:409- the instance is still running{"error":"Can't remove sequence in use."}Instance basic operations[ GET ]/api/v1/instances- list all instancesParametersNo parametersResponsesSuccessful operation code:200[{"id":"742d2713-7ab6-4cde-82f3-a7beabdd4e98","sequence":"bdef63db-d3a0-45c8-85db-e94ebb96097f"},{"id":"681c856e-dfa4-46a1-951d-47b27345552e","sequence":"bdef63db-d3a0-45c8-85db-e94ebb96097f"},{"id":"21f787ed-6b9e-4e9f-828e-afe428d84833","sequence":"bdef63db-d3a0-45c8-85db-e94ebb96097f"}][ GET ]/api/v1/instance/:id- show data of chosen instanceParametersNo parametersResponsesNameCodeDescriptionSuccessful operation200Returns JSON dataNot Found operation404For example if instance was already stopped{"created":"2021-10-29T16:08:36.524Z","started":"2021-10-29T16:08:38.701Z","sequenceId":"b0c02fdc-b05f-4f26-9d68-43a702eb7b44"}[ POST ]/api/v1/instance/:id/_stop- end instance gracefully and prolong operations or not for task completion​ParametersNameTypeDescriptionRequiredtimeoutnumberThe number of milliseconds before the Instance will be killed. Default: 7000ms.nocanCallKeepalivebooleanIf set to true, the instance will prolong the running. Default: false.noResponsesSuccessful operation code:200{"code":0,"type":"string","message":"string"}[ POST ]api/v1/instance/:id/_kill- end instance gracefully waiting for unfinished tasksParametersNo parametersResponsesAccepted operation code:202No body returned[ GET ]/api/v1/instance/:id/health- check status about instance healthParametersNo parametersResponsesSuccessful operation code:200{"cpuTotalUsage":529325247,"healthy":true,"limit":536870912,"memoryMaxUsage":16117760,"memoryUsage":14155776,"networkRx":1086,"networkTx":0,"containerId":"1c993c4ff774fac06185aa9554cf40c23b03e1479a7e0d14827708161b08ae51"}Instance advanced operation[ POST ]/api/v1/instance/:id/_event- send event to the InstanceParametersNameTypeDescriptionRequiredeventNamestringName of an eventtruemessagestringJSON formatted event payloadfalseResponsesContent-type:application/octet-stream[ GET ]/api/v1/instance/:id/event- get the data stream with the events from the InstanceParametersNo parametersResponsesContent-type:application/octet-stream[ GET ]/api/v1/instance/:id/once- get the last event sent by the InstanceParametersNo parametersResponsesContent-type:application/octet-stream[ POST ]/api/v1/instance/:id/input- send data to the input stream of the Instance to consume itParametersNo parametersResponsesNameCodeDescriptionSuccessful operation200-Not Acceptable operation406Instance expects the input to be provided from the Topic API[ GET ]/api/v1/instance/:id/output- get stream data from an instance and consume it through the endpointParametersNo parametersResponsesContent-type:application/octet-stream[ POST ]/api/v1/instance/:id/stdin​- process.stdinParametersNo parametersResponsesSuccessful operation code:200[ GET ]/api/v1/instance/:id/stdout- process.stdoutParametersNo parametersResponsesContent-type:application/octet-stream[ GET ]/api/v1/instance/:id/stderr- process.stderrParametersNo parametersResponsesContent-type:application/octet-stream[ GET ]/api/v1/instance/:id/log- stream all instance logsParametersNo parametersResponsesContent-type:application/octet-streamSuccessful operation code:2002021-11-19T16:12:22.948Zlog(Sequence)422021-11-19T16:12:23.949Zlog(Sequence)412021-11-19T16:12:24.950Zlog(Sequence)402021-11-19T16:12:25.951Zlog(Sequence)392021-11-19T16:12:26.952Zlog(Sequence)382021-11-19T16:12:27.952Zlog(Sequence)372021-11-19T16:12:28.953Zlog(Sequence)362021-11-19T16:12:29.953Zlog(Sequence)35Service Discovery: TopicsIf a given topic does not exist, Transform-Hub creates it and stores the sent data in the newly created topic. The data is stored in the topic until the data is not consumed (either by the Topic API or by the Instances subscribing to this topic). When the data are sent to the topic they are written to the returned stream.[ POST ]/api/v1/topics/:name​- sends data to the topicParametersNo parametersRequest HeadersHeaderTypeDescriptionDefaultRequiredx-end-streambooleanIf set totrue, then close topic stream after processing the request.falsenocontent-typetext,applicationSpecifies data type of this topicapplication/x-ndjsonnoSupported types:text/x-ndjson,application/x-ndjson,application/x-ndjson,text/plain,application/octet-streamResponsesNameCodeDescriptionSuccessful operation200data was sent with the header indicating the end of dataSuccessful operation202data was sent without the header indicating the end of data (default)[ GET ]/api/v1/topics/:name​- get data from the topicParametersNo parametersResponsesTopic data stream.Successful operation code:200{"source":"Twitter","id":"850006245121695778","content":"Natural wetlands make up ~30% of global total CH4 emissions","user":{"id":1234994945,"name":"Climate Change Conference","screen_name":"Climate Change",}}Some important linksScramjet, the company behindTransform HubTheScramjet Framework - functional reactive stream processing frameworkTheTransform Hub repo on githubYou can see theScramjet Transform Hub API docs hereYou can see theCLI documentation here, butsi helpshould also be quite effective.Don't forget to ⭐ this repo if you like it,subscribeto releases and keep visiting us for new versions and updates.You canopen an issue - file a bug report or a feature request hereLicense and contributionsThis module is licensed under AGPL-3.0 license.The Scramjet Transform Hub project is dual-licensed under the AGPL-3.0 and MIT licenses. Parts of the project that are linked with your programs are MIT licensed, the rest is AGPL.ContributionsWe accept valid contributions and we will be publishing a more specific project roadmap so contributors can propose features and also help us implement them. We kindly ask you that contributed commits are Signed-Offgit commit --sign-off.We provide support for contributors via test cases. If you expect a certain type of workflow to be officially supported, please specify and implement a test case inGherkinformat inbdddirectory and include it in your pull request. More info about our BDD test you will findhere.Help wanted 👩‍🎓🧑👱‍♀️The project need's your help! There's lots of work to do and we have a lot of plans. If you want to help and be part of the Scramjet team, please reach out to us,on discordor email us:opensource@scramjet.org.Donation 💸Do you like this project? It helped you to reduce time spent on delivering your solution? You are welcome to buy us a coffee ☕ Thanks a lot! 😉You can sponsor us on githubThere's also a Paypal donation link if you prefer that: