Phones have gyroscopes

Part 1: Multi-sensitive

Overview of sensors in modern smartphones

You can also make phone calls with a smartphone. This statement describes the functional scope of today's mobile devices well. You have a large number of sensors to record parameters from the environment. This article provides an overview of the most important antennas for perceiving the outside world. The following parts of the series are about technology and programming.

Current smartphones are literally “crammed full” with sensors. These include Ambient light, proximity, acceleration and rotation sensor (gyroscope), electromagnetic sensor, digital compass (magnetometer), fingerprint scanner and sensors for location determination (GPS). In addition, there are several cameras with which pictures and videos can now be recorded in impressive quality (Fig. 1).

Article series

Part 1: Overview of sensors in modern smartphones

Part 2: Geolocation - location with W-LAN, cellular network and GPS

Part 3: Further sensors under the microscope

camera

That's right, the cameras in a smartphone are also sensors. The quality of these cameras is one of the decisive factors when choosing a new smartphone. In the meantime, the quality of the recordings that can be made has also increased significantly; in many cases, cell phone cameras no longer have to fear comparison with the image quality of a simple compact camera. The size of the camera's sensor surface has a decisive influence on the image quality. The relationship between resolution and sensor size also plays an important role. A resolution of 20 million megapixels means that the sensor has the same number of light cells to take the picture. As a result, the stored images have the same number of pixels. Larger pixels (sensor surface) can absorb more light and have a positive effect on image quality. The structure of the image sensors also has an influence on the image quality. A distinction is made between Front Side Illumination (FSI), i.e. the wiring on the front between the incident light and the sensor surface, and Back Side Illumination (BSI), the wiring on the back of the sensors. With BSI, shadows on the sensor surface are avoided and a higher light sensitivity is achieved. The lens largely determines the optical properties of the camera [3].

Modern smartphones have i .d. Usually via several cameras. They have a front and a rear camera, the rear camera, usually with a much higher resolution, facing away from the viewer, while the front camera is used for video telephony or for taking self-portraits. With the help of dual cameras, two sensors and lenses are arranged side by side. One goal is to improve the image quality by calculating the image information from both cameras into one recording. Both lenses can also have different focal lengths (wide angle vs. zoom). With two cameras you can also add special effects to an image through the software, e.g. B. Sharpness and blurring effects.

microphone

The microphone is primarily used to make phone calls. Other applications that evaluate the audio signals from the microphone are conceivable. For example, as a voice recorder, for voice control of apps (artificial intelligence), for measuring ambient noise, as a simple baby monitor or for room monitoring. In addition to the built-in microphone, an external microphone, i. d. Usually connected with headphones.

Fingerprint sensor

Fingerprints have long been used to identify people. The reason for this is the uniqueness of a fingerprint. A few years ago, the fingerprint sensor in smartphones was still something special. Nowadays it is available in numerous devices and has almost become a matter of course. Fingerprint sensors are used for quick and easy unlocking of the devices by the user. Compared to other systems, e.g. PIN, password or pattern entry, the process is much simpler and significantly more secure. A distinction is made between semi-automatic and fully automatic fingerprint sensors. With the semi-automatic sensors, the finger has to be swiped over a sensor field in order to record the entire fingertip. What we know from smartphones today are fully automatic sensors where the fingertip can simply be placed on it. There are two common methods of scanning the fingerprint: optical and capacitive. Optical fingerprint sensors are the most common in smartphones. For identification, a finger is placed on the glass plate, which serves as a prism. The papillary ridges (bumps) of the fingerprint come into contact with the glass. The valleys (grooves) in between are filled with air and in this way do not touch the glass plate. A light source then illuminates the finger so that the prism reflects the light rays onto an image sensor. The reflection is disturbed where the rays hit the elevations. The image sensor creates a two-dimensional photo of the impression.

When the smartphone is unlocked, a comparison is made within seconds to determine whether the applied pattern matches the stored data. In the case of high-quality sensors, ultrasound or infrared is used in addition to the optical sensors. It is thus possible to recognize whether the owner of the fingerprint is alive in the identification. Properties such as pulse, blood circulation and thermal image are also examined.

The capacitive sensors are based on electrical charge. In this case there is a conductive silicon layer on top of the fingerprint scanner and a network of tiny capacitors underneath. These sit on a sensor chip. If the finger is placed on the silicon layer, the electrical charge changes. This only happens in the places where the finger grooves rest. Where there are no grooves, the load remains the same. With the help of an operational amplifier with an integrated circuit and an analog-to-digital converter, the whole thing is converted into a digital image of the fingerprint.

The fingerprint sensor is therefore used for identification. It cannot be assumed that it is completely free of errors, as is the case with other biometric methods. The biggest mistakes when using biometric procedures are false acceptance and false rejection. The false acceptance rate (FAR) is the probability with which an unauthorized person will be accepted based on similar biometric characteristics. The false rejection rate (FRR) is the probability with which an authorized person will be denied access because the matching requirements of biometric characteristics are handled too rigidly. The aim is to bring both parameters (FAR, FRR) into an acceptable relation to the security level [1].

GPS sensor

One of the reasons why your smartphone knows where you are is. on its built-in GPS receiver. Satellite navigation is one of the most useful and essential capabilities of a smartphone these days. GPS stands for Global Positioning System and was originally developed by the US Army in 1973. The official name is NAVSTAR GPS (Navigational Satellite Timing and Ranging - Global Positioning System). It is a global positioning system that was originally used in the military sector (box: “The GPS system”).

The GPS system

To find your way around the earth, there is the graticule. Geographical latitude and longitude are spherical coordinates with the help of which the position of a point on earth can be described exactly. The earth is divided into 180 degrees of latitude and 360 degrees of longitude. The circles of latitude run parallel to the equator. Count 90 parallels to the north and to the south. The size of the circles decreases with increasing proximity to the poles. The equator runs at right angles to the earth's axis. With an angle of 0 degrees, it is used as the starting point for calculating the parallels. The longitudes run through the north and south poles. There are 360 ​​of them. How do you get that number? The earth rotates around its axis once every 24 hours. The time difference from one degree of longitude to the next is 4 minutes. 360 degrees of longitude times 4 minutes results in 1,440 minutes, which corresponds to exactly 24 hours. Unlike the latitudes, the longitudes do not have a natural zero point. The meridian that runs through the London borough of Greenwich is the starting point for calculating longitudes. It divides the earth's surface into western and eastern hemispheres [2]. You can understand this with the help of Figure 2. The coordinates are measured in degrees (°), arc minutes (‚) and arc seconds („). One degree corresponds to 60 arc minutes, an arc minute in turn has 60 arc seconds. Can that also be more specific? Sure, using an example: The geographical position is 37 ° 49'N and 122 ° 29'W. Decimal notation is much more practical for processing using software. The values ​​are therefore converted. The above information becomes the latitude 37.816667 and the longitude -122.483333. The digits in front of the decimal point show the degree, the digits after the decimal point represent the converted values ​​for minutes and seconds. Positive signs indicate a north or east position and negative signs indicate a south or west position. The position mentioned here is that of the Golden Gate Bridge in San Francisco in a sexagesimal representation.

GPS is the most popular navigation system. However, there are alternatives that may be less well known:

  • GLONASS

  • LORAN-C

  • EutelTRACS

  • COMPASS-M

  • GALILEO

GALILEO will presumably be the satellite-based navigation system of the Europeans in the future.

Smartphones not only use GPS data to determine their whereabouts. They also calculate the position based on the data of the cellular network or the available WLAN access points in whose reception area they are located. These calculation alternatives are much simpler, faster and more energy efficient. However, there are disadvantages in terms of accuracy. The software (API of the operating system) can be used to specify the accuracy requirements and the system software independently decides which method is used for positioning. We will go into more detail in the second part of the series on this point and the more precise application of the location problem in connection with the software implementation.

Two further aspects of localization, possibly also GPS-based, still need to be addressed:

  1. privacy: Location data are considered personal data and are therefore particularly in need of protection according to the General Data Protection Regulation. Therefore, the "uncontrolled flow" of this data to the server of an app is repeatedly criticized. The misuse of the location data is often complained about. The advertising industry understandably has a special interest in such data. As a developer, you have a special responsibility in this regard. The Android and iOS systems always require the explicit consent of the user before access to the system's location API. The user can withdraw this authorization at any time, i. In other words, each time the service is used, the user's consent must be checked again.

  2. Background process: Location functions are often executed as a background process in an app. For example, with a tracking app, the position is determined at cyclical intervals and transmitted to a server (background) in the cloud via the network (cellular network). Due to this fact, when programming location functions, one must i. d. Usually also deal with the topic of using / programming background processes.

For better localization, the GPS sensor should be combined with a gyroscope sensor. Otherwise the GPS positioning is inaccurate.

Rotation sensor

The gyroscope or gyroscope was developed 200 years ago. In a gyroscope, a symmetrical top rotates in a movable bearing. This means that the bearing can move while the gyroscopic motion remains the same. In contrast to the first drafts of a gyroscope, the corresponding sensor is only a few millimeters small today. It is used in the smartphone to recognize its orientation. With the help of the Coriolis force and the tuning fork principle, the smartphone knows where the four corners of the device are in relation to each other in the room. It is therefore a sensor with which the exact alignment (position) can be determined. Many cell phones already have a built-in rotation sensor. Software can be used to check whether the device has a gyro compass. The gyroscope sensor in the smartphone also has a useful function when using navigation apps, because it enables even more precise position determination. The gyroscope is thus an extension of the GPS system. Therefore, the GPS and gyroscope sensors are usually installed together.

Magnetometer

The magnetometer is a digital compass that works and can be used like an analog compass. The magnetometer uses the earth's magnetic field to determine north direction. A mini version of a so-called fluxgate magnetometer is built into the smartphone. This consists of thin metal alloy plates to which a voltage is applied. The conduction path of the electrodes varies depending on the position of the smartphone in relation to the electromagnetic field of the earth. This is determined electrically and read out by the operating system. Often, however, the digital compass is not only used to determine the direction of the compass. For more precision, the data from the barometer for height measurement and the acceleration sensor to determine the spatial position are also used.

barometer

A barometer in the smartphone - why is that? This sensor turns the cell phone into a mobile weather station and gives the GPS module a head start. A ten micrometer thin silicon membrane, in which stretch marks are embedded, measures the air pressure. The size of the sensor is just two millimeters in width and two and a half millimeters in length. The barometer is also installed as a support for the acceleration sensor and can therefore also be helpful for the accuracy of the GPS system. By combining the measured values ​​from the sensors mentioned, a very precise location and position positioning can be carried out.

Ambient light sensor

The aim of the ambient light sensor is to measure the ambient light and to transfer this measured value to the system. The sensor consists of a photodiode with a color filter. The color temperature of light sources is measured. The screen brightness, contrast and color saturation are adjusted based on the measurement results.

Proximity sensor

The proximity sensor is used to check whether something is approaching the smartphone display. An infrared beam is used, which has a range of up to six centimeters. The sensor is i. d. Usually above the display next to the camera or earpiece. It is only half a millimeter thin and measures two by two millimeters in length and width. The proximity sensor is used to switch off the display during an active phone call. If the smartphone is brought to the ear, the sensor registers the reflection of the infrared rays through the cheek and the software switches off the display. This prevents the user from accidentally triggering an action by accidentally touching the touchscreen during the call. The touchscreen is reactivated when the smartphone is removed from the ear. The constantly active infrared beam also recognizes swiping and movement gestures of the hands.

Accelerometer

The acceleration sensor measures the acceleration of the device. In the smartphone, it has the task of recognizing the current position of the device and its changes. When turning the smartphone, the direction from which gravity acts on the device changes. This change is measured by a so-called accelerometer, which is only three by three millimeters in size. A silicon rod a few micrometers wide acts as a spring. The acceleration sensor detects the movement on three axes: the X-axis (left / right), the Y-axis (up / down) and the Z-axis (front / back). As the device moves, the position relative to a fixed electrode changes. The software then calculates the strength of the acceleration from the changing electrical capacitance. In this way it can be determined when a change in movement occurs. A typical application is rotating the smartphone and then aligning the screen.

Heart rate monitor

LEDs are used to measure the pulse. The change in color is measured by a photodiode when the blood flows slower or faster through the veins. The information is passed on to the system and the pulse rate is calculated from this value.

External sensors

Although modern smartphones already contain a large number of sensors, the measurement options can be expanded using external sensors. Some examples:

  • Gas meter for detecting carbon monoxide, alcohol and other toxic gases

  • Temperature sensor or non-contact infrared temperature sensor

  • Sensor for measuring the pressure

  • Sensor for measuring brightness and color intensity

  • Sensor for measuring radioactive radiation

  • a proximity switch with which cables or bars can be found in the wall

  • Sensors for measuring electrical quantities, for example voltage; this means that the smartphone can be used as a simple replacement for a voltmeter

  • Special sensors for measuring medical parameters

Such sensors can be purchased as external devices and linked to the smartphone. Data transfer works wirelessly i. d. Usually via Bluetooth. Coupling the smartphone with a smartwatch is also an extension of the sensors.

Sensor simulation during development and testing

Sensors usually process current input data from outside. During the development and testing of an app, emulators (Android) and simulators (iOS) are used very often. Apps that actively evaluate data from sensors must be triggered with real-time data as far as possible. This can only be measured on real physical devices. However, some measured values ​​can be simulated.

In the Android emulator, for example, you can set the desired target position manually by specifying the coordinates and thus simulate changing positions for the app that is executed on the emulator. A route (locations, connections) can also be simulated manually using tools, saved as a file and made available to the app during runtime. For some other sensors, too, data can be artificially generated via simulation to a certain extent for development and testing. Let's take a look at the Android emulator as an example. Start the emulator (installation via the Android SDK) and call up the extended menu (via the three points ...). In the Virtual Sensors section you can, for example, simulate the position of the smartphone (Fig. 3).

You can also make settings for other environmental values ​​(e.g. brightness, magnetic field, air pressure, temperature) manually (Fig. 4).

A limited simulation of the built-in cameras of a smartphone or tablet is also possible. You can load static images into the emulator for this. These images are then made available to your app in the emulator instead of the actual recording of the camera concerned (Fig. 5).

Overall, the sensor simulation options on Android are quite extensive, but cannot replace the test on a real physical device. A simulator under macOS is used for iOS apps. Here, too, you can set some values ​​for sensors manually.

Conclusion and outlook

Smartphones contain more and more sensors and are therefore able to measure a large amount of data from the environment or in interaction with the user. With the help of software, you can access the data from the sensors. The system software provides programming interfaces that allow easy use of the sensors without having to deal with the technical background in detail. However, a solid basic understanding of the limits of measurement technology in a smartphone, for example with regard to accuracy, is an important prerequisite for using it in your own app.

In the next issue, we will take a closer look at the possible applications, for example the personalization of services with the help of geolocation. A look at the programming practice with this sensor will not be missing either.

For more information on these and other IT topics, please visit http://larinet.com.

 

Our editorial team recommends: