search
Search
person_outline
By Van Schandevijl Wouter

Training with ML.NET and LightGBM on Synology with Docker
It just worked on Windows 😀

dotnet/machinelearning: ML.NET is an open source and cross-platform machine learning framework for .NET.

microsoft/LightGBM: A fast, distributed, high performance gradient boosting (GBT, GBDT, GBRT, GBM or MART) framework based on decision tree algorithms, used for ranking, classification and many other machine learning tasks.

After writing some code to fetch and mold the data, I was ready to train “my” AI, dubbed “PongRank” which was easy enough and all was good until I added a small WebApi to automate future syncing, retraining and predictions.

Category:

docker

Tags:

netsynologywar-story

Share this article on:

A heroic Linux penguin wielding a keyboard sword, battling giant digital dragons labeled 'lib_lightgbm.so' and 'libgomp.so.1', inside a dark futuristic server cave, glowing error messages flying through the air, Docker container shields, epic fantasy-tech crossover, humorous tone, digital fantasy art, high detail, vibrant lighting

Docker Training

As, once deployed, training failed with:

System.DllNotFoundException: Unable to load shared library ‘lib_lightgbm’ or one of its dependencies. In order to help diagnose loading problems, consider using a tool like strace. If you’re using glibc, consider setting the LD_DEBUG environment variable:
libgomp.so.1: cannot open shared object file: No such file or directory
/app/lib_lightgbm.so: cannot open shared object file: No such file or directory

It’s a conscious decision by the team to not include LightGBM by default, and that makes perfect sense since it is a 1gig dependency which is only used when using that trainer.

Installing LightGBM

A simple Google search got me to the LightGBM installation instructions:

git clone --recursive https://github.com/microsoft/LightGBM.git
cd LightGBM && mkdir build && cd build
cmake .. && make -j && make install

CMake Detour

The apt-get install cmake -y installed version 3.25.1 which was just not good enough for LightGBM!

CMake 3.28 or higher is required. You are running version 3.25.1

And now it was really starting to show that I’m very far from a Linux installation/compilation hero. After trying to build CMake for a few hours, I ended up installing pip to install CMake with.

apt-get install python3-pip -y
pip install cmake --upgrade --break-system-packages

# Check version:
cmake --version

The --break-system-packages obviously isn’t a very good sign but since we will be doing this in the build step of our Dockerfile, it’s fine 😅

Final Dockerfile

FROM mcr.microsoft.com/dotnet/sdk:8.0 AS build

RUN apt-get update
RUN apt-get install git clang-3.9 libunwind8 curl libomp-dev -y

RUN apt-get install python3-pip -y
RUN pip install cmake --upgrade --break-system-packages

RUN git clone --recursive https://github.com/microsoft/LightGBM.git
RUN cd LightGBM && mkdir build && cd build && cmake .. && make -j && make install

# Usual steps for building the .NET app
# dotnet restore/publish etc

FROM mcr.microsoft.com/dotnet/aspnet:8.0 as runtime

COPY --from=build /app /app
COPY --from=build /usr/local/lib/lib_lightgbm.so /usr/local/lib/lib_lightgbm.so
COPY --from=build /usr/lib/x86_64-linux-gnu/libgomp.so.1 /usr/lib/x86_64-linux-gnu/libgomp.so.1

ENTRYPOINT [ "dotnet", "/app/WebApi.dll" ]

I figured out which files needed to be copied with the following command:

# Enter running container
# (with lightgbm installed)
docker exec -it lightgbm bash

# Find location
find / -name 'lib_lightgbm.so' 2>/dev/null

# List dynamic dependencies
ldd /usr/local/lib/lib_lightgbm.so

And there we have it, a WebApi that creates the models!

Extras

Stuff that came into being during the making of this post

Other interesting reads