3D Printing a Smart Pillbox with Neural Network Image Recognition

Hospital pharmacists have their hands full, as they have to handle a wide variety of medications. They need to keep the medicine safe, and no matter how hard we may try, people aren’t perfect, so processes like these are always subject to human error. One way to solve this is by using a pill dispensing system, which can be fabricated with 3D printing. Five researchers from Taiwan published a paper, titled “Development of Smart Pillbox Using 3D Printing Technology and Convolutional Neural Network Image Recognition,” about the automated system they developed that makes it more accurate and simple to dispense medication.

“Safety in taking medicine is important in health care. In this study, we propose a complete concept of an active smart pillbox, which comprises a main control unit, a pill dispenser unit, and an application software (app) for the automatic dispensing of medicine. The smart pillbox employs convolutional neural network image recognition and 3D printing technology,” the abstract states.

They designed the system as a 3D-printed mechanism, which uses an Arduino-based platform to control its movement, and a mobile app to set the parameters via smartphone. The app sets the type of medication, and what time it should be taken, and the smartphone will then transmit the selected settings through Bluetooth to the main platform.

“The Arduino (master) sends the action commands through a MAX485 to the Arduino (slave). The Arduino (slave) receives the command and starts the action. In the Arduino (slave) action process, the main Arduino (master) continues transmitting the message to each Arduino (slave) and completes the actions for each Arduino (slave). After completely dispensing the pills, the main Arduino (master) returns the message to the smartphone to inform the user taking the medicine,” the researchers explained.

The pillbox system was designed in Solidworks, and it was 3D printed, along with the rotating gear.

At first, a button setting isused to operate the smart pillbox, but eventually, the mobile app will define which of the pills should be dispensed, and then set the medication commands for when they should be dropped.

“To increase the number of types of pills that can be stored, the Arduino (master) gives instructions to the slave through MAX485, sets the number of each type of pill, and operates the system,” they wrote. “The times when the pills should be dropped are set, according to the arranged kit number. Multiple times can be set, and the user is reminded to take the medicine at these different times.”

Once the amount of pills to be dispensed is set, the Arduino controller gives a command to start both the motor and the sensor. The motor gears drive the gears that connect to the rotating teeth, which then rotate the pill dispensary. Then, once the pill arrives at the chute, it’s dropped through, and a sensor counts the number of pills dispensed. For every pill dropped, a value on the display is reduced by one.

The other parts to this system are pill image recognition and the training model. Three processes are used to recognize a picture—identification, looking, and scaling down—and, in this case, a camera based on the popular convolutional neural network (CNN) is used to perform these and acquire images of the pills. These images are then transferred to the smartphone app for CNN recognition.

This CNN uses Googlenet architecture for model training, which has trouble achieving data diversity. But, as the object’s background in this system is limited, and features no miscellaneous identification, the researchers used a Siamese network, which is “often used for face recognition and model architecture training.”

“The principle is to capture the features of the neural network for image recognition. The final fully connected layer for classification is not entered, but the feature is entered as one (128, 1). The vector, also known as an embedding, then discards the photo of the identified item and the photo in the database is selected for the model to obtain a vector of (128, 1), and then the two vectors are compared,” the researchers wrote. “For training this model, first, we must define a loss function to measure the distance between embeddings. Here, we use the triplet-loss function; there will be three photos: the anchor and the positive and negative images of the anchor.”

As the data set is a picture of just one pill, in order to detect the individual pills, multiple images are processed so edge detection can be used to cut out and separate each one. The model finds the edge’s pixel-exposed part by following these steps:

• Eliminate noise and smooth image with a mask
• Calculate the horizontal and vertical gradients to find the boundary
• Find the contour

“Finding the contour: use the findtour function of cv2; with the CV_RETR_EXTERNAL parameter, only the outline of the outermost layer is taken. Input of square image: the cv2.boundingRect parameter is used and saved as an image file,” they stated.

There are still some issues to work out, such as excessive friction being generated if too many medicines are in the box, and the fact that the training model’s recognition rate with unseen data is low due to “lack of diversity of resources.”

“If we can accumulate more datasets in the future, the accuracy of the model can be increased, and the model can be compared with other models,” they stated.

But, because users can set their pill type and medication time with the app, it’s easy to use the pillbox system from the comfort of home – it actually provides a reminder once the parameters have been set, and will then dispense the right pills at the right time.

“The purpose of this study is to develop an intelligent automatic pillbox system on the basis of the concept of a pharmaceutical packaging system used in smart hospitals, a system that is tailored for personalized use. The development of the smartphone app provides a more convenient and safe medication system for patients’ homecare. It is not necessary to put the tablets one by one into a designated area according to the medication time of the user, which can reduce the unnecessary waste of time and exposure of tablets,” the researchers concluded.

Discuss this and other 3D printing topics at 3DPrintBoard.com or share your thoughts below.