Package: villaProductSdk
villaProductSdkInstallpip install villaProductSdkHow to useUploading a large amount of data using s3generate dummy data for testing#Dummy DatasampleProducts=[{'cprcode':'0171670','iprcode':'0171670','oprcode':'0171670','ordertype':'Y','pr_abb':'JIRAPAT YOUNG KALE 2','pr_active':'Y','pr_cgcode':'05','pr_code':'0171670','pr_dpcode':'19','pr_engname':'IRAPAT YOUNG KALE 200 G.','pr_ggcode':'057','pr_market':'JIRAPAT ยอดคะน้า 200 G.','pr_name':'JIRAPAT ยอดคะน้า 200 G.','pr_puqty':'1','pr_sa_method':'1','pr_sucode1':'CM845     ','pr_suref3':'A','prtype':'I','psqty':'1','pstype':'1'},{'cprcode':'0235141','iprcode':'0235141','oprcode':'0235141','ordertype':'Y','pr_abb':'EEBOO-PZCT3-PUZZLE','pr_active':'Y','pr_cgcode':'08','pr_code':'0235141','pr_dpcode':'19','pr_engname':'EEBOO,ANIMAL COUNTING PUZZLE_3ED,PZCT3','pr_ggcode':'113','pr_market':'eeboo,PUZZLE-PZCT3','pr_name':'EEBOO-PZCT3-ตัวต่อนับเลข ANIMAL COUNTING_3ED','pr_puqty':'1','pr_sa_method':'1','pr_sucode1':'CM1979    ','pr_suref3':'A','prtype':'I','psqty':'1','pstype':'1'},{'cprcode':'0217153','iprcode':'0217153','oprcode':'0217153','ordertype':'Y','pr_abb':'COCOA LOCO MILK CHOC','pr_active':'Y','pr_cgcode':'98','pr_code':'0217153','pr_dpcode':'28','pr_engname':'COCOA LOCO MILK CHOCOLATE OWL LOLLY 26G.','pr_ggcode':'003','pr_market':'COCOA LOCO MILK CHOCOLATE OWL','pr_name':'COCOA LOCO MILK CHOCOLATE OWL LOLLY 26G.','pr_puqty':'24','pr_sa_method':'1','pr_sucode1':'F1222     ','pr_suref3':'S','prtype':'I','psqty':'1','pstype':'1'},{'cprcode':'0182223','iprcode':'0182223','oprcode':'0182223','ordertype':'Y','pr_abb':'CIRIO PIZZASSIMO 400','pr_active':'Y','pr_cgcode':'06','pr_code':'0182223','pr_dpcode':'06','pr_engname':'CIRIO PIZZASSIMO 400G.','pr_ggcode':'004','pr_market':'CIRIO ซอสทำพิซซ่า 400 G.','pr_name':'CIRIO ซอสทำพิซซ่า 400 G.','pr_puqty':'12','pr_sa_method':'1','pr_sucode1':'2589      ','pr_suref3':'C','prtype':'I','psqty':'1','pstype':'1'},{'cprcode':'0124461','iprcode':'0124461','oprcode':'0124461','ordertype':'Y','pr_abb':'NEW CHOICE LYCHEE','pr_active':'Y','pr_cgcode':'02','pr_code':'0124461','pr_dpcode':'02','pr_engname':'NEW CHOICE LYCHEE','pr_ggcode':'003','pr_market':'NEW CHOICE กลิ่นลิ้นจี่','pr_name':'NEW CHOICE กลิ่นลิ้นจี่','pr_puqty':'12','pr_sa_method':'1','pr_sucode1':'695       ','pr_suref3':'A','prtype':'I','psqty':'1','pstype':'1'}]Create main class objectfromvillaProductSdk.productsimportProductSdksdk=ProductSdk(branch='dev-manual',user=USER,pw=PW)INFO:botocore.credentials:Found credentials in shared credentials file: ~/.aws/credentialsUpload lots of data using s3%%timeresult=sdk.updateWithS3(sampleProducts,invocationType='RequestResponse')resultINFO:root:bucket is input-product-bucket-dev-manual
INFO:root:using accelerate endpoint
INFO:root:data was saved to s3
INFO:root:data is saved to s3, invoking ingestion function
INFO:root:input to lambda is {'body': '{"key":"input-data-name"}', 'headers': {}, 'statusCode': 200}
INFO:root:lambdaResponse is {'body': '{"success":5,"failure":0,"skipped":0,"failureMessage":[],"timetaken(ms)":184.101}', 'statusCode': 200, 'headers': {'Access-Control-Allow-Headers': '*', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Methods': '*'}}


CPU times: user 66.4 ms, sys: 7.93 ms, total: 74.3 ms
Wall time: 1.08 s





{'success': 5,
 'failure': 0,
 'skipped': 0,
 'failureMessage': [],
 'timetaken(ms)': 184.101}Query Single Product%%timeresult=sdk.querySingleProduct('0171670')resultCPU times: user 5.76 ms, sys: 589 µs, total: 6.35 ms
Wall time: 71.3 ms





{'iprcode': '0171670',
 'cprcode': '0171670',
 'oprcode': '0171670',
 'ordertype': 'Y',
 'pr_abb': 'JIRAPAT YOUNG KALE 2',
 'pr_active': 'Y',
 'pr_cgcode': '05',
 'pr_code': '0171670',
 'pr_dpcode': '19',
 'pr_engname': 'IRAPAT YOUNG KALE 200 G.',
 'pr_ggcode': '057',
 'pr_market': 'JIRAPAT ยอดคะน้า 200 G.',
 'pr_name': 'JIRAPAT ยอดคะน้า 200 G.',
 'pr_puqty': '1',
 'pr_sa_method': '1',
 'pr_sucode1': 'CM845',
 'pr_suref3': 'A',
 'prtype': 'I',
 'psqty': '1',
 'pstype': '1'}All Queryresult=sdk.allQuery()sdk.printFirst(result)('cprcode',
 0    None
 Name: cprcode, dtype: object)Trigger s3 sync%%timeresponse=sdk.syncS3()responseERROR:root:error parsing body, perhaps there is no body in response      response is {'statusCode': 200, 'headers': {'Access-Control-Allow-Headers': '*', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Methods': '*'}}
Traceback (most recent call last):
  File "/mnt/efs/pip/villa-product-sdk/villaProductSdk/products.py", line 54, in returnLambdaResponse
    return Response.fromDict(lambdaResponse).body
  File "/home/ec2-user/SageMaker/.persisted_conda/python38/lib/python3.8/site-packages/awsSchema/apigateway.py", line 35, in fromDict
    body = json.loads(body),
ValueError: Expected object or value
ERROR:root:{'statusCode': 200, 'headers': {'Access-Control-Allow-Headers': '*', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Methods': '*'}}


CPU times: user 5.84 ms, sys: 3.6 ms, total: 9.44 ms
Wall time: 6.48 s