Defect detection in high-resolution imagery utilizing two-stage Amazon Rekognition Customized Labels fashions

Excessive-resolution imagery may be very prevalent in right this moment’s world, from satellite tv for pc imagery to drones and DLSR cameras. From this imagery, we will seize harm because of pure disasters, anomalies in manufacturing tools, or very small defects resembling defects on printed circuit boards (PCBs) or semiconductors. Constructing anomaly detection fashions utilizing high-resolution imagery could be difficult as a result of trendy pc imaginative and prescient fashions sometimes resize photographs to a decrease decision to suit into reminiscence for coaching and working inference. Decreasing the picture decision considerably implies that visible data regarding the defect is degraded or fully misplaced.

One strategy to beat these challenges is to construct two-stage fashions. Stage 1 fashions detect a area of curiosity, and Stage 2 fashions detect defects on the cropped area of curiosity, thereby sustaining ample decision for small detects.

On this publish, we go over how you can construct an efficient two-stage defect detection system utilizing Amazon Rekognition Customized Labels and examine outcomes for this particular use case with one-stage fashions. Be aware that a number of one-stage fashions are efficient even at decrease or resized picture resolutions, and others might accommodate giant photographs in smaller batches.

Answer overview

For our use case, we use a dataset of photographs of PCBs with synthetically generated lacking gap pins, as proven within the following instance.

We use this dataset to show {that a} one-stage strategy utilizing object detection leads to subpar detection efficiency for the lacking gap pin defects. A two-step mannequin is most popular, during which we use Rekognition Customized Labels first for object detection to establish the pins after which a second-stage mannequin to categorise cropped photographs of the pins into pins with lacking holes or regular pins.

The coaching course of for a Rekognition Customized Labels mannequin consists of a number of steps, as illustrated within the following diagram.

First, we use Amazon Easy Storage Service (Amazon S3) to retailer the picture information. The info is ingested in Amazon Sagemaker Jupyter notebooks, the place sometimes an information scientist will examine the pictures and preprocess them, eradicating any photographs which might be of poor high quality resembling blurred photographs or poor lighting situations, and resize or crop the pictures. Then information is cut up into coaching and take a look at units, and Amazon SageMaker Floor Fact labeling jobs are run to label the units of photographs and output a prepare and take a look at manifest file. The manifest information are utilized by Rekognition Customized Labels for coaching.

One-stage mannequin strategy

The primary strategy we take to figuring out lacking holes on the PCB is to label the lacking holes and prepare an object detection mannequin to establish the lacking holes. The next is a picture instance from the dataset.

We prepare a mannequin with a dataset with 95 photographs used as coaching and 20 photographs used for testing. The next desk summarizes our outcomes.

The ensuing mannequin has excessive precision however low recall, that means that after we localize a area for a lacking gap, we’re normally appropriate, however we’re lacking a whole lot of lacking holes which might be current on the PCB. To construct an efficient defect detection system, we have to enhance recall. The low efficiency of this mannequin could also be because of the defects being small on this high-resolution picture of the PCB, so the mannequin has no reference of a wholesome pin.

Subsequent, we discover splitting the picture into 4 or six crops relying on the PCB dimension and labeling each wholesome and lacking holes. The next is an instance of the ensuing cropped picture.

We prepare a mannequin with 524 photographs used as coaching and 106 photographs used for testing. We preserve the identical PCBs utilized in prepare and take a look at as the complete board mannequin. The outcomes for cropped wholesome pins vs. lacking holes are proven within the following desk.

Each precision and recall have improved considerably. Coaching the mannequin with zoomed-in cropped photographs and a reference to the mannequin for wholesome pins helped. Nonetheless, recall remains to be at 92%, that means that we might nonetheless miss 8% of the lacking holes and let defects go by unnoticed.

Subsequent, we discover a two-stage mannequin strategy during which we will enhance the mannequin efficiency additional.

Two-stage mannequin strategy

For the two-stage mannequin, we prepare two fashions: one for detecting pins and one for detecting if the pin is lacking or not on zoomed-in cropped photographs of the pin. The next is a picture from the pin detection dataset.

The info is just like our earlier experiment, during which we cropped the PCB into 4 or six cropped photographs. This time, we label all pins and don’t make any distinctions if the pin has a lacking gap or not. We prepare this mannequin with 522 photographs and take a look at with 108 photographs, sustaining the identical prepare/take a look at cut up as earlier experiments. The outcomes are proven within the following desk.

The mannequin detects the pins completely on this artificial dataset.

Subsequent, we construct the mannequin to make the excellence for lacking holes. We use cropped photographs of the holes to coach the second stage of the mannequin, as proven within the following examples. This mannequin is separate from the earlier fashions as a result of it’s a classification mannequin and might be centered on the slender activity of figuring out if the pin has a lacking gap.

We prepare this second-stage mannequin on 16,624 photographs and take a look at on 3,266, sustaining the identical prepare/take a look at splits because the earlier experiments. The next desk summarizes our outcomes.

Once more, we obtain excellent precision and recall on this artificial dataset. Combining the earlier pin detection mannequin with this second-stage lacking gap classification mannequin, we will construct a mannequin that outperforms any single-stage mannequin.

The next desk summarizes the experiments we carried out.

Inference pipeline

You should use the next structure to deploy the one-stage and two-stage fashions that we described on this publish. The next predominant parts are concerned:

For one-stage fashions, you possibly can ship an enter picture to the API Gateway endpoint, adopted by Lambda for any primary picture preprocessing, and path to the Rekognition Customized Labels skilled mannequin endpoint. In our experiments, we explored one-stage fashions that may detect solely lacking holes, and lacking holes and wholesome pins.

For 2-stage fashions, you possibly can equally ship a picture to the API Gateway endpoint, adopted by Lambda. Lambda acts as an orchestrator that first calls the thing detection mannequin (skilled utilizing Rekognition Customized Labels), which generates the area of curiosity. The unique picture is then cropped within the Lambda perform, and despatched to a different Rekognition Customized Labels classification mannequin for detecting defects in every cropped picture.

Conclusion

On this publish, we skilled one- and two-stage fashions to detect lacking holes in PCBs utilizing Rekognition Customized Labels. We reported outcomes for varied fashions; in our case, two-stage fashions outperformed different variants. We encourage prospects with high-resolution imagery from different domains to check mannequin efficiency with one- and two-stage fashions. Moreover, contemplate the next methods to broaden the answer:

• Sliding window crops on your precise datasets
• Reusing your object detection fashions in the identical pipeline
• Pre-labeling workflows utilizing bounding field predictions

In regards to the authors

Andreas Karagounis is a Information Science Supervisor at Accenture. He holds a masters in Pc Science from Brown College. He has a background in pc imaginative and prescient and works with prospects to unravel their enterprise challenges utilizing information science and machine studying.

Yogesh Chaturvedi is a Principal Options Architect at AWS with a spotlight in pc imaginative and prescient. He works with prospects to handle their enterprise challenges utilizing cloud applied sciences. Outdoors of labor, he enjoys climbing, touring, and watching sports activities.

Shreyas Subramanian is a Principal Information Scientist, and helps prospects by utilizing machine studying to unravel their enterprise challenges utilizing the AWS platform. Shreyas has a background in large-scale optimization and machine studying, and in the usage of machine studying and reinforcement studying for accelerating optimization duties.

Selimcan “Can” Sakar is a cloud-first developer and Options Architect at AWS Accenture Enterprise Group with a deal with rising applied sciences resembling GenAI, ML, and blockchain. When he isn’t watching fashions converge, he could be seen biking or taking part in the clarinet.