Question: I am trying to use the confluent-kafka python library to administer my cluster via a lambda function but the function fails with the error: "Unable to import module 'Test': No module named 'confluent_kafka.cimpl'"  My requirements.txt requests confluent-kafka  To create the zip file I moved my code to the site-packages location of the virtual env and zipped everything. Python Code: import confluent_kafka.admin import requests def lambda_handler(event, context): print("Hello World")  I am using the macOS 10.X. On Linux, I noticed that pip install creates a separate confluent_kafka.libs which does not get created on mac

Answer: I created the required layer and can verity that it works. The technique used includes docker tool described in the recent AWS blog:  How do I create a Lambda layer using a simulated Lambda environment with Docker?  Thus for this question, I verified it as follows:  Create empty folder, e.g. mylayer.  Go to the folder and create requirements.txt file with the content of   echo requests > requirements.txt echo confluent-kafka >> requirements.txt   Run the following docker command:  docker run -v "$PWD":/var/task "lambci/lambda:build-python3.8" /bin/sh -c "pip install -r requirements.txt -t python/lib/python3.8/site-packages/; exit"   Create layer as zip:  zip -r mylayer.zip python > /dev/null   Create lambda layer based on mylayer.zip in the AWS Console. Don't forget to specify Compatible runtimes to python3.8.  Test the layer in lambda using the following lambda function:   import confluent_kafka.admin import requests def lambda_handler(event, context): print(dir(confluent_kafka.admin)) print(dir(requests)) print("Hello World")  The function executes correctly: ['AdminClient', 'BrokerMetadata', 'CONFIG_SOURCE_DEFAULT_CONFIG', 'CONFIG_SOURCE_DYNAMIC_BROKER_CONFIG', 'CONFIG_SOURCE_DYNAMIC_DEFAULT_BROKER_CONFIG', 'CONFIG_SOURCE_DYNAMIC_TOPIC_CONFIG', 'CONFIG_SOURCE_STATIC_BROKER_CONFIG', 'CONFIG_SOURCE_UNKNOWN_CONFIG', 'ClusterMetadata', 'ConfigEntry', 'ConfigResource', 'ConfigSource', 'Enum', 'KafkaException', 'NewPartitions', 'NewTopic', 'PartitionMetadata', 'RESOURCE_ANY', 'RESOURCE_BROKER', 'RESOURCE_GROUP', 'RESOURCE_TOPIC', 'RESOURCE_UNKNOWN', 'TopicMetadata', '_AdminClientImpl', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__path__', '__spec__', 'concurrent', 'functools'] ['ConnectTimeout', 'ConnectionError', 'DependencyWarning', 'FileModeWarning', 'HTTPError', 'NullHandler', 'PreparedRequest', 'ReadTimeout', 'Request', 'RequestException', 'RequestsDependencyWarning', 'Response', 'Session', 'Timeout', 'TooManyRedirects', 'URLRequired', '__author__', '__author_email__', '__build__', '__builtins__', '__cached__', '__cake__', '__copyright__', '__description__', '__doc__', '__file__', '__license__', '__loader__', '__name__', '__package__', '__path__', '__spec__', '__title__', '__url__', '__version__', '_check_cryptography', '_internal_utils', 'adapters', 'api', 'auth', 'certs', 'chardet', 'check_compatibility', 'codes', 'compat', 'cookies', 'delete', 'exceptions', 'get', 'head', 'hooks', 'logging', 'models', 'options', 'packages', 'patch', 'post', 'put', 'request', 'session', 'sessions', 'ssl', 'status_codes', 'structures', 'urllib3', 'utils', 'warnings'] Hello World