Root Inoculation System

Overview

Root Inoculation System connects image-based plant analysis to robotic positioning for plant-microbe experiments. Developed for the Netherlands Plant Eco-phenotyping Centre (NPEC), the demonstrated system performs inoculation in an OT-2 robot simulation.

The engineering task was to connect several stages that use different representations: camera pixels, segmentation masks, plant locations, millimetres on a plate, and robot coordinates.

Problem

Manual root inoculation is difficult to scale consistently. Automating it requires both reliable root localisation and precise movement to the resulting target. A good segmentation result alone does not guarantee an accurate inoculation position.

Implementation

1. Detect the dishCrop and pad the plate image for patch-based processing.
2. Segment the imageRun the U-Net model on patches and reconstruct the mask.
3. Extract root tipsApply morphology and locate the lowest root point per plant.
4. Convert coordinatesMap pixel locations through plate dimensions into robot space.
5. Control movementUse PID or a trained SAC policy to approach each target.
6. Evaluate inoculationRecord target error and execution time in simulation.

The computer-vision experiments progress from binary root segmentation to multiclass U-Net segmentation of background, roots, shoots, and seeds. Preprocessing includes dish detection, cropping, padding, and 256 x 256 pixel patches. Postprocessing reconstructs the full image and applies rules for fragmented roots.

Calibration and Integration

The pipeline converts root-tip pixels to millimetres using the plate dimensions, then maps them into simulator coordinates. Plate origin, axis swapping, and offsets are explicit calibration parameters.

This conversion is a central integration challenge: a consistent coordinate offset can move every inoculation away from the intended tip even when segmentation looks correct. The runnable pipeline therefore supports debug crops, simulation GIFs, and benchmark CSV output to inspect intermediate and final behaviour.

Evaluation

Segmentation

The saved multiclass validation evaluation records root metrics of F1 0.8657, recall 0.8751, IoU 0.7632, and precision 0.8565. These assess pixel-level segmentation and should be interpreted separately from robot positioning error.

Controller Comparison

The repository documents 95 target attempts per controller, using a 1 mm positioning-error threshold. Error is the distance between the simulated inoculation position and the target root-tip position.

These are recorded simulator results, not measured performance on a physical robot. The timing comparison also reflects the implemented controllers and simulation settings.

Software and Limitations

The packaged CLI supports switching between PID and RL, headless execution, GIF recording, debug outputs, and benchmark export. Tests cover preprocessing, postprocessing, coordinate utilities, and controller behaviour.

Root-tip extraction uses assumptions about plant arrangement and downward growth. Calibration, fragmented masks, and unusual root geometry can affect end-to-end accuracy. The supplied SAC checkpoint has 2.2 million training steps; the documented comparison is not an exhaustive optimisation of either control approach.

Engineering Takeaway

The project’s main value is the integration from perception to action. Separate segmentation and positioning measurements make it possible to identify whether an error originates in the model, coordinate conversion, or controller.

Implementation References

• Pipeline, coordinate calibration, and benchmark documentation.
• Runnable pipeline and controllers.
• PID benchmark records and RL benchmark records.
• Archived segmentation experiments.
• Notebook containing the saved multiclass validation metrics.

Tech stack: Python, TensorFlow/Keras, U-Net, image processing, PyBullet OT-2 simulation, PID control, Soft Actor-Critic, Pytest.