View source for Optical sorting

{{Short description|Automated sorting of solid products using cameras and/or lasers.}}
{{multiple issues|
{{advert|date=June 2015}}
{{more footnotes needed|date=June 2015}}
{{more citations needed|date=June 2015}}
}}
'''Optical sorting''' (sometimes called '''digital sorting''') is the automated process of [[sorting]] solid products using [[cameras]] and/or [[lasers]].

Depending on the types of sensors used and the software-driven intelligence of the [[image processing]] system, optical sorters can recognize an object's color, size, shape, structural properties and chemical composition.<ref>{{Cite book|title=Computer vision technology for food quality evaluation|date=2008|publisher=Elsevier/Academic Press|others=Sun, Da-Wen.|isbn=978-0-12-373642-0|edition=1st|location=Amsterdam|oclc=228148344}}</ref> The sorter compares objects to user-defined accept/reject criteria to identify and remove defective products and foreign material (FM) from the production line, or to separate product of different grades or types of materials.

Optical sorters are in widespread use in the [[food industry]] worldwide, with the highest adoption in processing harvested foods such as potatoes, fruits, vegetables and nuts where it achieves non-destructive, 100 percent inspection in-line at full production volumes.{{citation needed|date=May 2023}} The technology is also used in [[pharmaceutical manufacturing]] and [[nutraceutical]] manufacturing, tobacco processing, [[waste recycling]] and other industries. Compared to manual sorting, which is subjective and inconsistent, optical sorting helps improve product quality, maximize throughput and increase yields while reducing labor costs.<ref>{{Cite book|title=Nondestructive testing of food quality|date=2008|publisher=Blackwell Pub./IFT Press|others=Irudayaraj, Joseph, 1961-, Reh, Christoph.|isbn=978-0-470-38828-0|edition=1st|location=Ames, Iowa|oclc=236187975}}</ref>

== History ==
Optical sorting is an idea that first came out of the desire to automate industrial sorting of agricultural goods like fruits and vegetables.<ref name=":3">{{Cite web|url=https://en.unitec-group.com/about-us/|title=About us|website=Unitec Group|language=en-US|access-date=2020-04-02|archive-date=2017-03-01|archive-url=https://web.archive.org/web/20170301152235/https://en.unitec-group.com/about-us/|url-status=live}}</ref> Before automated optical sorting technology was conceived in the 1930s, companies like Unitec were producing wooden machinery to assist in the mechanical sorting of fruit processing.<ref name=":3" /> In 1931, a company known as “the Electric Sorting Company” was incorporated and began the creation of the world’s first color sorters, which were being installed and used in Michigan’s bean industry by 1932.<ref name=":4">{{Cite web|url=https://www.satake-usa.com/what-is-optical-sorting.html|title=Optical Sorting Technology Explained - Satake USA|website=www.satake-usa.com|access-date=2020-04-02|archive-date=2019-10-24|archive-url=https://web.archive.org/web/20191024141319/https://www.satake-usa.com/what-is-optical-sorting.html|url-status=live}}</ref> In 1937, optical sorting technology had advanced to allow for systems based on a two-color principle of selection.<ref name=":4" /> The next few decades saw the installation of new and improved sorting mechanisms, like gravity feed systems and the implementation of optical sorting in more agricultural industries.<ref name=":5">{{Cite web|url=https://www.satake-usa.com/history-of-satake-vision-systems.html|title=Optical Sorting Systems - Satake USA|website=www.satake-usa.com|access-date=2020-04-02|archive-date=2014-11-27|archive-url=https://web.archive.org/web/20141127053945/https://www.satake-usa.com/history-of-satake-vision-systems.html|url-status=live}}</ref>

In the late 1960s, optical sorting began to be implemented to new industries beyond agriculture, like the sorting of ferrous and non-ferrous metals.<ref name=":6">{{Cite web|url=http://www.mssoptical.com/about/history/|title=The History Of MSS {{!}} Optical Sorting & Recycling Technology|language=en-US|access-date=2020-04-02|archive-date=2019-04-26|archive-url=https://web.archive.org/web/20190426100017/http://www.mssoptical.com/about/history/|url-status=live}}</ref> By the 1990s, optical sorting was being used heavily in the sorting of solid wastes.<ref name=":6" />

With the large technological revolution happening in the late 1990s and early 2000s, optical sorters were being made more efficient via the implementation of new optical sensors, like CCD, [[Ultraviolet|UV]], and IR cameras.<ref name=":5" /> Today, optical sorting is used in a wide variety of industries and, as such, is implemented with a varying selection of mechanisms to assist in that specific sorter’s task.

== The sorting system ==
[[File:SortingPotatoStrips.jpg|thumb|Optical sorting achieves non-destructive, 100 percent inspection in-line at full production volumes.]]

In general, optical sorters feature four major components: the feed system, the optical system, image processing software, and the separation system.<ref>{{Cite book|title=Detecting foreign bodies in food|date=2004|publisher=CRC Press|others=Edwards, M. C. (Michael Charles)|isbn=1-85573-839-2|location=Boca Raton|oclc=56123328}}</ref> The objective of the feed system is to spread products into a uniform monolayer so products are presented to the optical system evenly, without clumps, at a constant velocity. The optical system includes lights and sensors housed above and/or below the flow of the objects being inspected. The image processing system compares objects to user-defined accept/reject thresholds to classify objects and actuate the separation system. The separation system — usually compressed air for small products and mechanical devices for larger products, like whole potatoes — pinpoints objects while in-air and deflects the objects to remove into a reject chute while the good product continues along its normal trajectory.

The ideal sorter to use depends on the application. Therefore, the product's characteristics and the user's objectives determine the ideal sensors, software-driven capabilities and mechanical platform.

== Sensors ==
Optical sorters require a combination of lights and sensors to illuminate and capture images of the objects so the images can be processed. The processed images will determine if the material should be accepted or rejected.

There are camera sorters, laser sorters and sorters that feature a combination of the two on one platform. Lights, cameras, lasers and laser sensors can be designed to function within visible light wavelengths as well as the [[infrared]] (IR) and [[ultraviolet]] (UV) [[spectrum]]s. The optimal wavelengths for each application maximize the contrast between the objects to be separated. Cameras and laser sensors can differ in spatial resolution, with higher resolutions enabling the sorter to detect and remove smaller defects.

=== Cameras ===
[[File:SortingGreenBeans.jpg|thumb|Shape sorting enables the detection of same-color defects and foreign material.]]
Monochromatic cameras detect shades of gray from black to white and can be effective when sorting products with high-contrast defects.

Sophisticated color cameras with high color resolution are capable of detecting millions of colors to better distinguish more subtle color defects. [[Trichromatic]] color cameras (also called three-channel cameras) divide light into three bands, which can include red, green and/or blue within the visible spectrum as well as IR and UV. The interaction of different materials with parts of the electromagnetic spectrum make these contrasts more evident than how they appear to the naked human eye.

Coupled with intelligent software, sorters that feature cameras are capable of recognizing each object's color, size and shape; as well as the color, size, shape and location of a defect on a product. Some intelligent sorters even allow the user to define a defective product based on the total defective surface area of any given object.

=== Lasers ===
While cameras capture product information based primarily on material reflectance, lasers and their sensors are able to distinguish a material's structural properties along with their color. This structural property inspection allows lasers to detect a wide range of organic and inorganic foreign material such as insects, glass, metal, sticks, rocks and plastic; even if they are the same color as the good product.

Lasers can be designed to operate within specific wavelengths of light; whether on the visible spectrum or beyond.<ref>{{Cite patent|title=Scattered/transmitted light information system|country=US|number=5675419|pubdate=1997-10-07|inventor1-last=van den Bergh|inventor1-first=Herman|inventor2-last=Lane|inventor2-first=Marvin|inventor3-last=Mallon|inventor3-first=John}}</ref> For example, lasers can detect chlorophyll by stimulating fluorescence using specific wavelengths; which is a process that is very effective for removing foreign material from green vegetables.<ref>{{Cite journal|last1=Zheng|first1=Hong|last2=Lu|first2=Hongfei|last3=Zheng|first3=Yueping|last4=Lou|first4=Heqiang|last5=Chen|first5=Cuiqin|date=2010-12-01|title=Automatic sorting of Chinese jujube (Zizyphus jujuba Mill. cv. 'hongxing') using chlorophyll fluorescence and support vector machine|url=http://www.sciencedirect.com/science/article/pii/S0260877410003754|journal=Journal of Food Engineering|language=en|volume=101|issue=4|pages=402–408|doi=10.1016/j.jfoodeng.2010.07.028|issn=0260-8774|access-date=2013-12-03|archive-date=2015-09-24|archive-url=https://web.archive.org/web/20150924165645/http://www.sciencedirect.com/science/article/pii/S0260877410003754|url-status=live|url-access=subscription}}</ref>

=== Camera/laser combinations ===
Sorters equipped with cameras and lasers on one platform are generally capable of identifying the widest variety of attributes. Cameras are often better at recognizing color, size and shape while laser sensors identify differences in structural properties to maximize foreign material detection and removal.

=== Hyperspectral Imaging ===
[[File:Hyperspectral image of "sugar end" potato strips.jpg|thumb|Hyperspectral image of "sugar end" potato strips shows invisible defects.]]
Driven by the need to solve previously impossible sorting challenges, a new generation of sorters that feature [[multispectral]] and [[hyperspectral imaging]] '''Optical Sorters'''.<ref>{{Cite web|url=http://www.vision-systems.com/articles/print/volume-17/issue-2/features/spectral-imaging-sorts-sugar-end-defects.html|title=StackPath|website=www.vision-systems.com|date=February 2012 |access-date=2020-03-24|archive-date=2013-12-08|archive-url=https://web.archive.org/web/20131208035535/http://www.vision-systems.com/articles/print/volume-17/issue-2/features/spectral-imaging-sorts-sugar-end-defects.html|url-status=live}}</ref>

Like trichromatic cameras, multispectral and hyperspectral cameras collect data from the electromagnetic spectrum. Unlike trichromatic cameras, which divide light into three bands, hyperspectral systems can divide light into hundreds of narrow bands over a continuous range that covers a vast portion of the electromagnetic spectrum. This opens the door for more detailed analysis that leads to a more consistent product. Using IR alone might detect some defects, but combining it with a broader range of the spectrum makes it more effective. Compared to the three data points per pixel collected by trichromatic cameras, hyperspectral cameras can collect hundreds of data points per pixel, which are combined to create a unique [[spectral signature]] (also called a fingerprint) for each object. When complemented by capable software intelligence, a hyperspectral sorter processes those fingerprints to enable sorting on the chemical composition of the product. This is an emerging area of [[chemometrics]].

== Software-driven intelligence ==
Once the sensors capture the object's response to the energy source, image processing is used to manipulate the raw data. The image processing extracts and categorizes information about specific features. The user then defines accept/reject thresholds that are used to determine what is good and bad in the raw data flow. The art and science of image processing lies in developing [[algorithms]] that maximize the effectiveness of the sorter while presenting a simple user-interface to the operator.

Object-based recognition is a classic example of software-driven intelligence. It allows the user to define a defective product based on where a defect lies on the product and/or the total defective surface area of an object. It offers more control in defining a wider range of defective products. When used to control the sorter's ejection system, it can improve the accuracy of ejecting defective products. This improves product quality and increases yields.

New software-driven capabilities are constantly being developed to address the specific needs of various applications. As computing hardware becomes more powerful, new software-driven advancements become possible. Some of these advancements enhance the effectiveness of sorters to achieve better results while others enable completely new sorting decisions to be made.

== Platforms ==
The considerations that determine the ideal platform for a specific application include the nature of the product – large or small, wet or dry, fragile or unbreakable, round or easy to stabilize – and the user's objectives. In general, products smaller than a grain of rice and as large as whole potatoes can be sorted. Throughputs range from less than 2 metric tons of product per hour on low-capacity sorters to more than 35 metric tons of product per hour on high-capacity sorters.

=== Channel sorters ===
The simplest optical sorters are channel sorters, a type of [[color sorter]] that can be effective for products that are small, hard, and dry with a consistent size and shape; such as rice and seeds. For these products, channel sorters offer an affordable solution and ease of use with a small footprint. Channel sorters feature monochromatic or color cameras and remove defects and foreign material based only on differences in color.

For products that cannot be handled by a channel sorter – such as soft, wet, or nonhomogeneous products – and for processors that want more control over the quality of their product, freefall sorters (also called waterfall or gravity-fed sorters), chute-fed, sorters or belt sorters are more ideal. These more sophisticated sorters often feature advanced cameras and/or lasers that, when complemented by capable software intelligence, detect objects' size, shape, color, structural properties, and chemical composition.

=== Freefall and chute-fed sorters ===
Freefall sorters inspect product in-air during the freefall and chute-fed sorters stabilize product on a chute prior to in-air inspection. The major advantages of freefall and chute-fed sorters, compared to belt sorters, are a lower price point and lower maintenance. These sorters are often most suitable for nuts and berries as well as frozen and dried fruits, vegetables, potato strips and seafood, in addition to waste recycling applications that require mid-volume throughputs.

=== Belt sorters ===
[[File:SortingPeas.jpg|thumb|Optical sorters can function within visible light wavelengths as well as the IR and UV spectrums.]]
Belt sorting platforms are often preferred for higher capacity applications such as vegetable and potato products prior to canning, freezing or drying. The products are often stabilized on a conveyor belt prior to inspection. Some belt sorters inspect products from above the belt, while other sorters also send products off of the belt for an in-air inspection. These sorters can either be designed to achieve traditional two-way sorting or three-way sorting if two ejector systems with three outfeed streams are equipped.

=== ADR systems ===
A fifth type of sorting platform, called an automated defect removal (ADR) system, is specifically for potato strips (French fries). Unlike other sorters that eject products with defects from the production line, ADR systems identify defects and actually cut the defects from the strips. The combination of an ADR system followed by a mechanical nubbin grader is another type of optical sorting system because it uses optical sensors to identify and remove defects.

=== Single-file inspection systems ===
The platforms described above all operate with materials in bulk; meaning they do not need the materials to be in a single-file to be inspected. In contrast, a sixth type of platform, used in the pharmaceutical industry, is a single-file optical inspection system. These sorters are effective in removing foreign objects based on differences in size, shape and color. They are not as popular as the other platforms due to decreased efficiency.

=== Mechanical graders ===
For products that require sorting only by size, mechanical grading systems are used because sensors and image processing software is not necessary. These mechanical grading systems are sometimes referred to as sorting systems, but should not be confused with optical sorters that feature sensors and image processing systems.

== Practical usage ==

=== Waste and recycling ===
Optical sorting machines can be used to identify and discard manufacturing waste, such as metals, drywall, cardboard, and various plastics.<ref>{{Cite web|url=https://www.epa.gov/hwgenerators/typical-wastes-generated-industry-sectors|title=Typical Wastes Generated by Industry Sectors|last=US EPA|first=OLEM|date=2015-05-26|website=US EPA|language=en|access-date=2020-04-02|archive-date=2015-06-22|archive-url=https://web.archive.org/web/20150622070322/https://www.epa.gov/hwgenerators/typical-wastes-generated-industry-sectors|url-status=live}}</ref> In the metal industry, optical sorting machines are used to discard plastics, glass, wood, and other non-needed metals.<ref name=":0">{{Cite web|url=http://www.mssoptical.com/service/optical-sorting-innovation/|title=Optical Sorting Applications And Sorting Machines {{!}} MSS|language=en-US|access-date=2020-04-02|archive-date=2020-03-24|archive-url=https://web.archive.org/web/20200324011047/http://www.mssoptical.com/service/optical-sorting-innovation/|url-status=live}}</ref> The plastic industry uses optical sorting machines to not only discard various materials like those listed, but also different types of plastics. Optical sorting machines discard different types of plastics by distinguishing resin types. Resin types that optical sorting machines can identify are: PET, HDPE, PP, PVC, LDPE, and others. Most recyclables are in the form of bottles. <ref>[https://www.mswsorting.com/Waste-Sorting/Optical-Sorter.html MSWsorting]</ref> <ref name=":0" />

Optical sorting also aids in recycling since the discarded materials are stored in bins. Once a bin is full of a given material, it can be sent to the appropriate recycling facility.<ref>{{Cite web|url=https://www.waste360.com/commentary/look-optical-sorting|title=A Look at Optical Sorting|date=2015-09-10|website=Waste360|language=en|access-date=2020-04-02|archive-date=2015-10-13|archive-url=https://web.archive.org/web/20151013175416/https://www.waste360.com/commentary/look-optical-sorting|url-status=live}}</ref> Optical sorting machines’ ability to distinguish between resin types also aids in the process of [[plastic recycling]] because there are different methods used for each plastic type.<ref>{{Citation|title=Plastic recycling|date=2020-04-01|url=https://en.wikipedia.org/w/index.php?title=Plastic_recycling&oldid=948453827|work=Wikipedia|language=en|access-date=2020-04-02|archive-date=2023-08-13|archive-url=https://web.archive.org/web/20230813195740/https://en.wikipedia.org/w/index.php?title=Plastic_recycling&oldid=948453827|url-status=live}}</ref>

=== Food and drink ===
In the coffee industry, optical sorting machines are used to identify and remove underdeveloped coffee beans called quakers; quakers are beans that contain mostly carbohydrates and sugars.<ref name=":1">{{Cite web|url=https://counterculturecoffee.com/blog/optical-sorting-qa|title=Optical Sorting Q&A|date=2018-09-21|website=Counter Culture Coffee|language=en-US|access-date=2020-04-02|archive-date=2020-03-24|archive-url=https://web.archive.org/web/20200324011931/https://counterculturecoffee.com/blog/optical-sorting-qa|url-status=live}}</ref>  A more accurate calibration offers a lower total number of defective products.<ref name=":1" /> Some coffee companies like Counter Culture use these machines in addition to pre-existing sorting methods in order to create a better tasting cup of coffee.<ref name=":1" /> One limitation is that someone has to program these machines by hand to identify defective products.<ref name=":1" />

However, this science is not limited to coffee beans; food items such as mustard seeds, fruits, wheat, and hemp can all be processed through optical sorting machines.<ref>{{Cite web|url=https://www.intelseed.ca/|title=Intel Seed Ltd. {{!}} Grain and Forage Seed Manitoba {{!}} Seed Cleaning and Conditioning Manitoba {{!}} HOME|website=www.intelseed.ca|access-date=2020-04-02|archive-date=2014-10-26|archive-url=https://web.archive.org/web/20141026102101/https://www.intelseed.ca/|url-status=live}}</ref>

In the wine manufacturing process, grapes and berries are sorted like coffee beans.<ref name=":2">{{Cite book|last1=Beyerer|first1=Jürgen|url=https://books.google.com/books?id=4F4x5T6Yc04C&q=optical+sorting+grapes&pg=PA115|title=OCM 2013 - Optical Characterization of Materials - conference proceedings|last2=León|first2=Fernando Puente|date=2013|publisher=KIT Scientific Publishing|isbn=978-3-86644-965-7|language=en|access-date=2020-12-07|archive-date=2023-08-13|archive-url=https://web.archive.org/web/20230813195725/https://books.google.com/books?id=4F4x5T6Yc04C&q=optical+sorting+grapes&pg=PA115|url-status=live}}</ref> Grape sorting is used to ensure no unripe/green parts to the plant are involved in the wine making process.<ref name=":2" /> In the past, manual sorting via sorting tables was used to separate the defective grapes from the more effective grapes.<ref name=":2" /> Now, mechanical harvesting provides a higher effectiveness rate compared to manual sorting.<ref name=":2" /> At different points in the line, materials are sorted out via several optical sorting machines.<ref name=":2" /> Each machine is looking for various materials of differing shapes and sizes.<ref name=":2" />

The berries or grapes can then be sorted accordingly using a camera, a laser, or a form of LED technology with regard to the shape and form of the given fruit. The sorting machine then discards any unnecessary elements.<ref>{{Cite journal|last1=Lafontaine|first1=Magali|last2=Freund|first2=Maximilian|last3=Vieth|first3=Kai-Uwe|last4=Negara|first4=Christian|date=2013-12-01|title=Automatic Fruit Sorting by Non-Destructive Determination of Quality Parameters Using Visible/Near Infrared to Improve Wine Quality: I. Red Wine Production|journal=NIR News|volume=24|issue=8|pages=6–8|language=en|doi=10.1255/nirn.1403|s2cid=95941922}}</ref>

<ref>{{Citation|title=Optical Inspection, Measurement and Sorting Machine for Fasteners, Washer, Screw, Bol, Nut, Precious Parts, Insert, O Ring etc.|url=https://vesindia.org/products/machine-vision-system/vision-optical-measurement-machine-system.aspx|access-date=2023-03-16|archive-date=2023-03-16|archive-url=https://web.archive.org/web/20230316082049/https://vesindia.org/products/machine-vision-system/vision-optical-measurement-machine-system.aspx|url-status=live}}</ref>

<ref>{{Citation|title=Optical Inspection and Sorting Machine for Fasteners, Precision machine parts, Turned components, Precision Brass components, Precision automotive components, Plastic & Rubber Parts.|url=https://www.varadautomation.com/optical-inspection-sorting-machine-glass-disc.html|access-date=2024-05-20}}</ref>
== See also ==
{{portal|Technology|Food}}
* [[Food grading]]
* [[Food safety]]
* [[Food technology]]
* {{section link|Recycling|Sorting}}

== References ==
{{Reflist}}

[[Category:Industrial machinery]]
[[Category:Food processing]]
[[Category:Image processing]]
[[Category:Recycling]]
[[Category:Applications of computer vision]]