Troubleshooting: Export platform-specific Python package using Docker

Python + Docker

0. Scenario

- A Python application deployed to AWS Lambda (Amazon Linux runtime)
- A Lambda layer created (external Python package .zip file uploaded from Windows 8 local machine)

Issue:
When trying to run the Lambda application, import error is thrown

Failed to import the Cloud Firestore library for Python. Make sure to install the "google-cloud-firestore" module

Python code:

from firebase_admin import firestore

Cause:
Python package grpcio compiles a platform specific dynamic module.
In my case, cygrpc.cp37-win_amd64.pyd (for Windows)
https://stackoverflow.com/questions/51391458/firebase-on-aws-lambda-import-error

1. Install Docker Toolbox

- download and run DockerToolbox-19.03.1.exe
https://github.com/docker/toolbox/releases

2. Launch and configure Docker

- click the Docker Quickstart Terminal Shortcut
https://www.tutorialspoint.com/docker/docker_installation.htm

- wait until the configuration completed and you will see an interactive shell

3. Download Docker image for Python

$> docker run -it python:3.7 bash

After the image is downloaded, you will be logged in as a root user in the Debian container

4. Install external Python library

$> pip3 install YOUR_LIBRARY

5. Check the installed package location

$> pip --version

You package files should be inside /usr/local/lib/python3.7/site-packages

6. Detach/exit from the running container
As we run the container with -it option before, press Ctrl+p, Ctrl+q, then Ctrl+c to detach it. Or you can just type exit command to exit it.

List the container info:

$> docker ps -a

Notice that the container ID is a8ce71a18715 here

Status if detached: Up, Status if exited: Exited

7. Save your new image

$> docker commit <container_id> new_image_name:tag_name(optional)

e.g. docker commit a8ce71a18715 layer:1.0

8. Copy the package from the container to your local file system

$> docker cp <container_id>:src_path dest_path

e.g. docker cp a8ce71a18715:/usr/local/lib/python3.7/site-packages c:\\Users\\user\\Desktop\\site-packages

Pay attention to the escape character in git bash!
https://stackoverflow.com/questions/46866478/docker-cp-command-not-allowed

9. Copy and paste the platform-specific file to your Python package folder
In my case, cygrpc.cpython-37m-darwin.so (for Linux)

Then package it into .zip and deploy to Lambda layer!

Other useful Docker commands:

Run your image as container:
$> docker run -it <image_name>:<tag_name> bash
Re-attach running container:
$> docker attach <container_id>
Start exited container:
$> docker start <container_id>
Stop/Kill running container:
$> docker stop <container_id>
or
$> docker kill <container_id>
Remove stopped container:
$> docker rm <container_id>
Remove all stopped containers:
$> docker container prune
List docker images:
$> docker images
Remove docker image:
$> docker rmi <image_id>

Useful Linux commands:

Install vi:
$> apt-get update
$> apt-get install vim
Check running OS in Docker container:
$> cat /etc/os-release