1.CCD (Pixel) and Image Processing Basics
1.1 CCD image sensor
A digital camera has almost the same structure as that of a conventional (analog) camera, but the difference is that a digital camera comes equipped with an image sensor called a CCD. The image sensor is similar to the film in a conventional camera and captures images as digital information, but how does it convert images into digital signals?
The CCD stands for a Charge Coupled Device, which is a semiconductor element that converts images into digital signals. It is approx. 1 cm in both height and width, and consists of small pixels aligned like a grid.
When taking a picture with a camera, the light reflected from the target is transmitted through the lens, forming an image on the CCD. When a pixel on the CCD receives the light, an electric charge corresponding to the light intensity is generated. The electric charge is converted into an electric signal to obtain the light intensity (concentration value) received by each pixel.
This means that each pixel is a sensor that can detect light intensity (photo diode(二极管)) and a 2 million-pixel CCD is a collection of 2-million photo diodes.
A photoelectric sensor can detect presence/absence of a target of a specified size in a specified location. A single sensor, however, is not effective for more complicated applications such as detecting targets in varying positions, detecting and measuring targets of varying shapes, or performing overall position and dimension measurements. The CCD, which is a collection of hundreds of thousands to millions of sensors, greatly expands possible applications including the four major application categories on the first page.
1.what is a conventional (analog) camera ?
2.A single sensor, however, is not effective for more complicated applications such as detecting targets in varying positions ?
1.2 Use of pixel data for image processing
The last section of this guide briefly details the method in which light intensity is converted into usable data by each pixel and then transferred to the controller for processing.
Individual pixel data (In the case of a standard black-and-white camera)：
In many vision sensors, each pixel transfers data in 256 levels (8 bit) according to the light intensity. In monochrome(单色) (black & white) processing, black is considered to be “0” and white is considered to be “255”, which allows the light intensity received by each pixel to be converted into numerical data This means that all pixels of a CCD have a value between 0 (black) and 255 (white). For example, gray that contains white and black, exactly half and half, is converted into “127”.
1.3 An image is a collection of 256-level data
Image data captured with a CCD is a collection of pixel data that make up the CCD, and the pixel data is reproduced as a 256-level contrast data.
As in the example above, image data is represented with values between 0 and 255 levels per pixel. Image processing is processing that finds features on an image by calculating the numerical data per pixel with a variety of calculation methods as shown below.
1.4 Example:Stain / Defect inspection
The average intensity of a segment (4 pixels x 4 pixels) is compared with that of the surrounding area. Stains are detected in the red segment in the above example.
1.5 Summary of CCD and image processing basics
Machine vision systems can detect areas (No. of pixels), positions (point of change in intensity), and defects (change in amount of intensity) with 256-level intensity data per pixel of a CCD image sensor. By selecting systems with higher pixel levels can higher speeds, you can easily expand the number of possible applications for your industry.
2. How Do CCDs Capture Images?
Most cameras used in machine-vision systems employ charge-coupled devices (CCDs) to convert an image into the electrical signals a computer can capture. But how does that conversion take place, and what are the implications of the conversion techniques for camera users? To understand the capabilities of various cameras, you need to understand how different types of CCDs work.
CCDs were developed as data-storage devices in the early 1970s, but the devices’ light sensitivity at visible wavelengths led researchers to use them as sensors. The CCDs used in cameras are simply ICs that provide regular arrays of individual photodetector elements, or sensors, that measure only a few tens of microns(微米) on a side. CCDs manufacturers place the devices in standard IC packages and cover them with a transparent window that passes visible light (Fig. 1).
Figure 1. Rectangular CCDs come in standard IC packages with transparent lids that let visible light shine on the thousands of sensors on the device’s surface. (Courtesy of Kodak.)
The real work in a CCD takes place at individual sensors made up of light-sensitive MOS capacitors. Each capacitor(电容) forms an individual sensor for a picture element, or pixel. Some photons reflect off(反射) the surface and a few others may get absorbed deep in the silicon substrate（硅基片）, but most hit the MOS capacitor and create hole-electron pairs(空穴-电子对). As long as light reaches the MOS capacitor, electrons continue to collect in a charge well (Fig. 2) formed by potentials applied to the CCD. In effect, the sensors integrate the light they receive.
Figure 2. A capacitor formed as a MOS device produces electron-hole pairs when “hit’’ by a photon. The electrons accumulate in a potential well formed by the charge on the gate.
2.1 Shift Those Electrons
After the electrons have accumulated at each sensor for a set exposure time, the CCD must convert these “packets’’ of electrons into a useful electrical signal. The simplest CCD array provides a single line of sensors, a linear array, that finds use in a linescan camera.1 Each sensor connects to a parallel-in, serial out（串行输出 ） shift register (Fig. 3).
Control circuitry(电路) on the CCD loads the shift register with the electrons from all sensors simultaneously. Then, the shift register quickly moves the electron packets to a charge-to-voltage amplifier（电荷电压放大器）. By converting the amplifier’s voltage to a digital value in sync with the shift register’s clock, an analog-to-digital converter (ADC) can produce a digital value corresponding to the light intensity at each sensor in the array. (A camera provides buffering circuits. A frame grabber(帧接收器) would not connect directly to a CCD.)
CCD manufacturers employ several techniques that use multi-phase clocks to shift the charges from place to place in an array. Suffice(有能力) it to say that the shift register acts like a bucket brigade that moves charges from place to place, each in its own packet like a line of people passing along buckets of water. Usually the clock, driver, timing, and logic circuits that control a CCD array exist as separate devices. In most cases, it proves impractical to integrate them into the CCD. Depending on the circuits used in a camera, the camera may produce a raw video output signal, a standard video output signal such as RS-170/NTSC, or digitized signals. (See “For Further Reading’’)
Figure 3. A linear array transfers charge packets to two shift registers that transport the charges to two amplifiers. The amplifiers convert the charges to voltages that, after further signal conditioning, a frame grabber can read.
2.2 Two Registers Double the Speed
To speed the charge-shifting process, CCD manufacturers may provide two shift registers, one on each side of a linear array. The register on one side connects to the even-numbered sensors and the other shift register connects to the odd-numbered sensors. The CCD loads both shift registers simultaneously and shifts out the charges two pixels at a time—one(第一时间 ) per shift register. As a result, the CCD puts out the video information twice as fast as a single shift-register device. External circuitry can recombine the two voltages to furnish a single video output. Or a frame grabber could digitize the two signals and combine the results to reform a vector of pixel intensities.
In an area-array CCD, which captures an entire image at a time, retrieving the charges from the array of sensors proves a bit more difficult. The array must provide shift registers for each column and a horizontal shift register to shift the charge packets to the amplifier (Fig 4.) In a full-frame array, the control circuits move the charges in unison down each column from one MOS capacitor to the next.After a row’s worth of charges get shifted out, charges from the row above get moved down and out to the amplifier. In this way, the charges from successive rows get put out as voltages from the CCD.
A full-frame CCD requires a shutter to block light during the entire read-out operation. As charges get shifted down a column they move from sensor to sensor, and they will gather additional electrons from illuminated sensors. The added electrons act to smear(污迹) the image, thus the need for a shutter.
By incorporating the sensors right in the shift register, a full-frame CCD requires no extra space for shift registers. Thus, these devices work well in applications that require high-resolution images because sensors can cover almost all of the CCD’s surface with no gaps between sensors. Almost all the light that hits the full-frame CCD gets converted to electrons. Speed can suffer, though, particularly in large arrays. Keep in mind that the CCD must shift out all the video information before it can start to acquire a new image.
To circumvent(避免) this time-delay problem, some CCD manufacturers offer frame-transfer devices that integrate two adjacent and equivalent arrays of sensors on one CCD. One array uses its MOS capacitors to gather light. The second array, which is not light sensitive, provides memory cells for the image information.
After the array acquires an image, circuitry quickly shifts the charge packets for the entire image, or frame, into the memory. The memory can then put out the image information using shift registers as described for a full-frame device. Although the CCD rapidly transfers a complete image to the memory, the charges still pass through the other sensors on their way, thus integrating some light during transfer. As a result, if a camera without a shutter uses a frame-transfer device, users can expect some image smearing. Also, because frame-transfer devices require twice the area of full-frame devices, they cost more. But the frame-transfer devices work well in applications that require high-speed image acquisition.
You may find cameras that offer split frame-transfer CCDs. These CCDs simply divide the imaging array in half and provide a memory for each half. The memories may be divided again so they feed four shift registers and produce four output voltages. Dividing the array into several smaller portions complicates the circuitry slightly and it means that the camera’s circuits must cope with several simultaneous video-output signals. Either a frame grabber or the camera must reconstruct an array of intensity values from the multiple outputs.
Figure 4. A full-frame CCD moves the charge packets down each column into a shift register along the bottom edge of a device. The shift register delivers the charges to an amplifier for conversion to a voltage.
2.3 Separate Registers from Sensors
A third type of CCD places columns of shift registers between the sensors. After acquiring an image, the array quickly transfers all of the charges into the column shift registers. These registers then feed a shift register that transfers the charges to an amplifier. As soon as the camera transfers an image to these interline shift registers, it can start to acquire another image.
By separating the shift registers from the sensors, the design eliminates smearing. But the shift registers in these interline devices take space. As a result, the sensors have spaces between them and they convert only a fraction of the light that reaches the CCD into useful information.
No matter which CCD comes in a camera, all CCDs are subject to an effect called “blooming,’’ which occurs when a bright light shines on a sensor or sensors. The bright light causes the sensor to quickly fill, or saturate, its charge well. The excess electrons can flow into adjacent wells and saturate them, too. When saturation occurs, the image obtained from the CCD show a large white splotch(斑点) at the place of bright illumination. The size of the splotch determines how much blooming occurred in the CCD.
CCD manufacturers can overcome blooming. One technique provides a CCD with an electronic overflow that operates much like an overflow drain in a bathroom sink. The electron well is set up so any excess electrons combine with holes, thus moving the electrons into the CCD substrate(底层). A second approach lets users adjust the ratio of photon hits to electrons produced. Producing fewer electrons per burst of photons will reduce blooming.
Some CCDs and cameras let you control anti-blooming effects. Anti-blooming lets the CCD withstand much more light than it could otherwise before saturating charge wells. Of course, you could reduce the aperture of a camera, or add a filter to reduce light levels, too.
2.4 Watch Out for Noise
You might think that when no light shines on a CCD’s sensors, they generate no electrons. Unfortunately, thermal and electrical effects always produce electrons. CCD manufacturers refer to these currents as dark currents. When the shutter is closed, dark currents cause some electrons to get trapped in charge wells. When the shutter opens to capture an image, the dark-current electrons add to those produced by any incident light, thus adding noise to the video information.
The noise reduces the CCD’s dynamic range, a measure of its ability to accurately resolve changes in light levels. The ratio of the CCD’s maximum output signal to the output signal due only to dark current represents the signal-to-noise(信噪比) ratio (in dBs) of the device. A CCD with a dynamic range of 48 dB indicates that a CCD can resolve light levels across its range with an accuracy of about one part in 256 (28), while a dynamic range of 60 dB indicates an accuracy of one part in 1024 (210). A camera manufacturer may improve dynamic range by electrically cooling the CCD, usually with a Peltier device. But such a camera usually will end up in a research lab.
The CCD manufacturers specify dynamic ranges for their devices, so camera manufacturers should provide this information, too. The dynamic range information will help you determine how many shades of gray a camera will accurately detect. The higher the dynamic range, the more shades of gray(灰度梯度). When comparing cameras, be sure that the manufacturers specify the conditions at which they performed the noise tests, usually a temperature and a light sensitivity (for example,
25 8C and 0.5 lux2,3). T&MW
what process of the amplifier ?
Is there a triangle in figure 4;
why need a shutter?
By separating the shift registers from the sensors, the design eliminates smearing?
The blooming phenomenon makes the high gray level luminance data more smooth for its brightness light overflow to its adjacent ?
1.CCD (Pixel) and Image Processing Basics
2.How Do CCDs Capture Images?