Package: discomputify-P-Y-R-O-B-O-T
DISCOMPUTIFYPROJECT LINK GITHUBPROJECT LINK PYPIA library written purely in python to do distributed computing using CYPHER-PROTOCOL.Many times the task is very large and only single thread or single computer is not sufficient to do the computation, specially when the tasks are independent of each other or partially depend on each other.Thus we need to run the programs in parallel maner to complete the computation process faster and effeciently.COMPATIBLITYREADPY works with Linux, OS-X, and Windows (OS INDEPENDENT).Supported on python 3.10 and aboveINSTALLATIONInstall it using pip.GotoPYPIUSAGECONFIGURATIONFor configuration of server and client need a configuration file named discomputify_conf.jsonCONFIG PARAMETERSRECV_BUFFERandTRANSMISSION_BUFFERare parameters both should have same corresponding value for both server and client configuration.ENCRYPTION_KEYandDECRYPTION_KEYare parameters which ensure encryption of data sent over connection, see CYPHER-PROTOCOL documentation for more information.TIMEOUTis the time after which a connection is closed or reset from client side or server side.RECV_CHUNK_SIZEandTRANSMISSION_CHUNK_SIZEare the parameters that control chunk size for file transfersDEBUG1andDEBUG2control the debug levels of CYPHER_PROTOCOLINPUT_PATHandOUTPUT_PATHare parameters that specify the relative path for storing and retrieving  input and output filesFTP_NODE,QUERY_NODE,MASTER_NODE,SHARED_MEM_NODEspecify the addresses where the particiler service nodes are runningFTP_PORT,QUERY_PORT,MASTER_PORT,SHARED_MEM_PORTspecify the ports at which these particular services are running or will be runningFTP_BASE_PATHis to set the base path of files in FTP so that there is no clash between namespaces and files are perfectly organisedCOMPUTING_NODESis an array of addresses of computing services availableCOMP_NODEis used to set address of computing node while starting a computing serviceExample discomputify_conf is given in CONF_FILES folder, consider that for more details.STARTING A COMPUTING SERVICEPut discomputify_conf.json in a directory and run the commandpython3-mDISCOMPUTIFYcomputing_nodeSTARTING A FTP SERVICEPut discomputify_conf.json in a directory and run the commandpython3-mDISCOMPUTIFYftp_nodeSTARTING A MASTER NODE SERVICEPut discomputify_conf.json in a directory and run the commandpython3-mDISCOMPUTIFYmaster_nodeUSAGE OF CLIENT FOR DISTRIBUTED COMPUTINGFor starting computation, create a directory and put all the input files for the program in a directory as specified in discompitify_conf.jsonthen add the program that you want to run with all of its dependencies in the directory, but make sure that the starting poin of your program should be main.pySee an example of main.py in test_scriptsHere is the example for using the CLIENT_NODE for computingfromDISCOMPUTIFY.CLIENT_NODE.client_nodeimportCLIENT_NODEimporttimeif__name__=="__main__":CLIENT_NODE_OBJ=CLIENT_NODE()CLIENT_NODE_OBJ.set_process_id("1684340918413322")print("UPLOADING FILES")CLIENT_NODE_OBJ.upload_files("qwe")print("UPLOADING COMPLETE")print("STARTING COMPUTING")print(CLIENT_NODE_OBJ.start_computing())print("COMPUTING STARTED")time.sleep(1)print("GETTING NODE LOAD")print(CLIENT_NODE_OBJ.get_node_load())print("GETTING PROCESS STATUS")print(CLIENT_NODE_OBJ.get_process_status())#print("TERMINATING PROCESS")#CLIENT_NODE_OBJ.stop_computing()#print("PROCESS TERMINATED")print("GETTING PROCESS STATUS")print(CLIENT_NODE_OBJ.get_process_status())print("GOT PROCESS STATUS")print("GETTING FILES")CLIENT_NODE_OBJ.download_files()print("DOWNLOADED FILES")#print("CLEARING DATA")#print(CLIENT_NODE_OBJ.clear_data())#print("CLEARED DATA")