Position sensor architecture for cameras

A camera system may include circuitry for measuring the positions of an optics assembly (e.g., one or more lenses) and/or an image sensor of the camera system. The circuitry may include analog circuits comprising a first and a second position sensors to produce a first and a second sensor signals based on a first magnetic field and a second magnetic field respectively. The magnetic fields may have the same or different polarities detectable by the position sensors. The position sensors may be coupled in parallel in the same or reverse directions to produce a combined sensor output. The circuitry may determine position information for the optics assembly and/or the image sensor based on the combined sensor output. The camera system may use the position information as a feedback signal to control the position of the optics assembly (e.g., for autofocus) and/or the position of the image sensor (e.g., for optical image stabilization (OIS)).

BACKGROUND

Technical Field

This disclosure relates generally to a camera and more specifically to measurement of the position information of an optics assembly (e.g., one or more lenses) and/or an image sensor of the camera in mobile devices.

Description of the Related Art

The advent of mobile multipurpose devices such as smartphones, tablet or pad devices has resulted in a need for bigger lenses such as ultra- or super-wide lenses for integration in cameras of the devices. Some cameras may incorporate optical image stabilization (OIS) mechanisms that may sense and react to external excitation/disturbance by adjusting location of the optical lens on the X and/or Y axis in an attempt to compensate for unwanted motion of the lens. Furthermore, some cameras may incorporate an autofocus (AF) mechanism whereby the object focal distance can be adjusted to focus an object plane in front of the camera at an image plane to be captured by the image sensor. In some such AF mechanisms, the optical lens is moved as a single rigid body along the optical axis of the camera to refocus the camera.

In addition, high image quality is easier to achieve in small form factor cameras if lens motion along the optical axis is accompanied by minimal parasitic motion in the other degrees of freedom, for example on the X and Y axes orthogonal to the optical (Z) axis of the camera. Thus, some small form factor cameras that include autofocus mechanisms may also incorporate optical image stabilization (OIS) mechanisms that may sense and react to external excitation/disturbance by adjusting location of the optical lens and/or the image sensor on the X and/or Y axis in an attempt to compensate for unwanted motion of the lens. In such systems, knowledge of the position of the lens and/or the image sensor is useful.

DETAILED DESCRIPTION

Various embodiments described herein relate to measurement of the positions of an optics assembly (e.g., one or more lenses) and/or an image sensor of a camera system of a mobile device. In some embodiments, the position measurement may be implemented based on measurement circuitry including a plurality of position sensors. The position sensors may be configured to measure the positions of the optics assembly and/or the image sensor based on measuring one or more magnetic fields produced by respective magnets. For instance, the sensor signal, e.g., an output voltage, of a position sensor may vary with the strength of the magnetic field detectable by the position sensor. The magnetic field may be produced with a position sensor (or position probe) magnet. For a given position sensor magnet, the strength of the magnetic field changes with the relative distance and/or angle between the position sensor magnet and the position sensor. Thus, the position of the position sensor magnet may be determined based on the value of the sensor signal from the position sensor. For instance, the camera system may include a position sensor magnet affixed to and thus moveable with the optics assembly (or the image sensor), whilst the position sensor may be attached to a stationary platform. Thus, as the optics assembly (or the image sensor) moves relative to the stationary platform, e.g., in an autofocus movement (or optical image stabilization (OIS) movement), the position of the position sensor magnet and the optics assembly (or the image sensor) may be measured with the position sensor.

In some embodiments, the measurement circuitry may comprise a plurality of position sensors including a first and second position sensors configured to produce a first sensor signal and a second sensor signal based on a first magnetic field and a second magnetic field (produced respectively by a first and second position sensor magnets) respectively. In some embodiments, the first and second magnetic fields may have the same or different polarities detectable by the first and second position sensors. In some embodiments, the first and second position sensors may be coupled (directly or indirectly) in parallel in the same or reverse directions to produce a combined sensor output. In some embodiments, the measurement circuitry may comprise a signal processing circuit to determine the position information, e.g., the position, distance, and/or angle, of the optics assembly and/or the image sensor of the camera system based on the combined sensor output. In some embodiments, the position information, e.g., the position, distance, and/or angle, may be determined relative to the corresponding position sensors.

In some embodiments, the first and second position sensor magnets are configured to be moveable with the optics assembly and/or the image sensor. As described above, the position sensor magnets may be affixed to the optics assembly and/or the image sensor and thus become moveable with respect to stationary position sensors. In some embodiments, the first and second position sensors may be configured to be moveable whilst the position sensor magnets may be affixed to a stationary platform. For instance, the position sensors may be attached to moveable part(s) of the camera system, e.g., the optics assembly and/or the image sensor.

In some embodiments, the position sensor magnets may be positioned at the same side of the optics assembly and/or the image sensor. In some embodiments, they may be placed at different locations, e.g., at diagnostically opposite positions around the camera system. In some embodiments, the position sensor(s) may include one or more of a Hall sensor, a tunneling magnetoresistance (TMR) sensor, a giant magnetoresistance (GMR) sensor, an anisotropic magnetoresistance (AMR) sensor, and so on.

In some embodiments, the camera system may include an actuator, e.g., a voice coil motor (VCM) actuator, to move the optics assembly and/or the image sensor based on the position measurement of the measurement circuitry. For instance, the actuator may take the position measurement of the optics assembly as a feedback signal and control the position of the optics assembly relative to the image sensor in an autofocus movement. Moreover, the actuator may move the image sensor relative to the optics assembly in an OIS movement based on the position measurement of the image sensor. In some embodiments, the camera system may use dedicated position sensor magnets, separate from other magnets, for measuring the position of the optics assembly and/or the image sensor. In some embodiments, the camera system may use the autofocus and/or OIS control magnets as the position sensor magnets to measure the position of the optics assembly and/or the image sensor, without separate/dedicated autofocus and/or OIS position sensor magnets.

FIGS.1-4illustrate embodiments of an example actuator assembly in which embodiments of position measurements as described herein may be applied. As one of skill in the art will readily ascertain in light of having read the included disclosure, a wide variety of configurations of position sensors and position sensor magnets fulfill differing design goals in different embodiments without departing from the scope and intent of the present disclosure. As one of skill in the art will readily ascertain in light of having read the included disclosure, a wide variety of configurations of actuator fulfill differing design goals in different embodiments without departing from the scope and intent of the present disclosure. For example, while the embodiments shown herein reflect voice coil motor actuators, one of skill in the art will readily understand that different actuators, including no-magnetic motorized actuators such as rotary motors or piezo-electric actuators, can be used with embodiments without departing from the scope and intent of the present disclosure.

FIG.1Ais a schematic diagram illustrating example measurement of autofocus position and/or image sensor position of a camera system, according to some embodiments. In this example shown inFIG.1A, a camera system may include actuator100having optics105that may comprise one or more optical elements110and one or more image sensors (not shown). In some embodiments, the one or more optical elements100may include one or more flat or curved optic lenses. The optics assembly105may be mounted to a lens carrier105. In some embodiments, the actuator100may include one or more coils and position control magnets that may interact with each other electromagnetically to produce motive forces (e.g., Lorentz forces) moving the lens carrier (and thus the optics assembly105) relative to the image sensor. For instance, the one or more actuators may move the optics assembly105close or further from the image sensor along the Z (optical) axis to implement autofocus. In some embodiments, the one or more actuator may also move the image sensor relative to the optics assembly105. For instance, the one or more actuators may move the image sensor relative to the optics assembly105on a XY plane along X-Y axis that are orthogonal to the Z (optical) axis to implement the optical image stabilization (OIS).

In some embodiments, the camera system may include multiple position sensor magnets (or probe magnets) and associated position sensors to measure the position of the optics assembly105(hereinafter named “autofocus position”) and/or the position of the image sensor. The camera system may further take the position measurement as a feedback signal to control the movement of the optics assembly105and/or the image sensor in autofocus and OIS. In this example shown inFIG.1A, the actuator100of the camera system may include a first position sensor magnet120and a second position sensor magnet125. The position sensor magnets120and125may be mounted upon the lens carrier115and thus become moveable with the optics assembly105relative to the image sensor, according to some embodiments. The actuator100may further include a first position sensor130and a second position sensor135, both of which are positioned proximate the position sensor magnets120and125respectively. Thus, the position sensor130may detect predominantly the magnetic field produced by the position sensor magnet120, whilst the position sensor135may interact predominantly with the magnetic field from the position sensor magnet125. In some embodiments, the first and second position sensors130and135may include one or more of a Hall sensor, a tunneling magnetoresistance (TMR) sensor, a giant magnetoresistance (GMR) sensor, and/or an anisotropic magnetoresistance (AMR) sensor. In some embodiments, the position sensor magnets120and125may be arranged such that they create magnetic fields of different polarities detectable by the position sensors130and135. For instance, as shown inFIG.1A, the position sensor magnet120may be arranged to have its south pole towards the position sensor130, and the position sensor magnet125may be placed to expose its north pole close to the position sensor135. Thus, the position sensor130may detect predominantly the magnetic field from the south pole of the position sensor magnet120, whilst the position sensor135may detect predominantly the magnetic field from the north pole of the position sensor magnet125. The above arrangement may deduce sensor signals v1and v2, e.g., output voltages, from respective P-N terminals of the position sensors130and135in opposite directions. For instance, the sensor signal v1from the position sensor130may be a positive voltage whilst the sensor signal v2from the position sensor135may be a negative voltage. As the optics assembly105(and/or position sensor magnets120and125) move along the Z (optical) axis (and/or X-Y axes orthogonal to the Z axis), the magnetic fields detected by the position sensors130and135may vary and thus the sensor signals from the position sensors130and135may change according to the autofocus position of the optics assembly105relative to the image sensor. For instance, when the optics assembly105moves further away from the image sensor, the magnetic fields detected by the positions sensors130and135may become weaker, and the sensor signals from the position sensors130and135may reduce (e.g., the absolute values or amplitudes of v1and v2become smaller). Alternatively, when the optics assembly105moves closer relative to the image sensor, the magnetic fields detected by the position sensors130and135may become stronger, and the sensor signals from the position sensors130and135may increase (e.g., the absolute values or amplitudes of v1and v2become larger). In some embodiments, the position sensors130and135may be coupled in parallel in reverse directions, as shown inFIGS.1A(and2A), to produce a combined sensor output vpn. For instance, given the example inFIG.1A, the sensor signal v1may be a positive voltage across the P-N terminals of the position sensor130, whilst the sensor signal v2may be a negative voltage across the P-N terminals of the position sensor135. In this example, the positions sensors130and135may be coupled in parallel in reverse directions with the P-terminal of the position sensor130electrically connected to the N-terminal of the position sensor135, the N-terminal of the position sensor130electrically connected to the P-terminal of the position sensor135, and the combined sensor output vpn created across the P-N terminals of the position sensor130. In some embodiments, the position sensors130and135may be positioned spatially in substantially the same direction, such that the parallel coupling of the position sensors130-135in reverse directions may cancel the effect of an external magnetic field on the combined sensor output. Note that when the arrangement of the position sensor magnets120and125(e.g., the polarities of their respective magnetic fields) with respect to the position sensors130and135changes, the connection between the position sensors130and135may be adjusted accordingly. In some embodiments, the position information, e.g., the autofocus position, may be determined based on the combined sensor output vpn, which may be further used as a feedback signal to control the position of the optics assembly105in autofocus movements.

FIG.1Bis a schematic diagram illustrating another example measurement of autofocus position and/or image sensor position of a camera system, according to some embodiments. In this example, the actuator100bof the camera may include a first and a second position sensor magnets120band125b. In some embodiments, the position sensor magnets120band125bmay be mounted on the lens carrier115. The actuator100bmay include a first and a second position sensors130band135b, positioned proximate the position sensor magnets120band125b, respectively. The position sensor magnets120b-125band position sensor130b-135bmay be configured such that the respective magnetic fields of the position sensor magnets120b-125bdetectable substantially by the position sensors130b-135bmay have the same polarity. For instance, as shown inFIG.1B, both the position sensor130band135bmay primarily expose to the magnetic fluxes from north to south poles of the position sensor magnets120band125b, respectively. As the position sensors130b-135bmay see the magnetic fields of the same magnetic polarity, the position sensors130b-135bmay produce sensor signals, e.g., v1and v2across respective P-N terminals, with the same electrical polarity. In some embodiments, the position sensors130band135bmay be coupled, directly or indirectly through electrical connections, in parallel (in the same direction) to produce a combined sensor output. For instance, as shown inFIG.1B, the P terminal of the position sensor130bmay be coupled to the P terminal of the position sensor135b, and the N terminal of the position sensor130bmay be coupled to the N terminal of the position sensor135b, to produce the combined sensor output vpn. In some embodiments, the position sensors130and135may be positioned spatially in substantially opposite directions, such that the parallel coupling of the position sensors130-135in the same direction may cancel the effect of an external magnetic field on the combined sensor output.

As described above, the positions information, e.g., the position, distance, and/or angle, of autofocus optics assembly and/or image sensor of a camera may be measured by positions sensors working together with position sensor magnets. In some embodiments, the position sensor may measure the magnetic field of the position sensor magnetics and produce an analog position measurement signals, such as an output voltage. Generally, the raw output signal from a position sensor is a small value. Thus, the analog measurement signal may be amplified by an amplifier circuit, converted to digital signals by an A/D converter, and processed by a processor to calculate the position in the digital domain. As described above, the camera may use multiple position sensors to determine a position in order to eliminate common mode noises caused by an external magnetic field and improve measurement accuracy. In such configurations, the analog signals from the multiple sensors may be processed (e.g., by amplifier and A/D converter) individually in separate analog channels before they may be analyzed by the processor in combination to calculate the position. Each of the individual analog channels may be vulnerable to noises. Further, such configurations may require a large number of analog devices (e.g., amplifiers and A/D converters) and device I/O capacities. Issues like this may be addressed by combining the analog measurement signals first and then sending the combined analog signal to the digital domain for processing.

FIG.2Ais a simplified schematic diagram showing example measurement circuitry for measuring the autofocus position and/or image sensor positions of a camera system, according to at least some embodiments. In the example shown inFIG.2A, measurement circuitry200may comprise a plurality of position sensors including a first position sensor215and a second position sensor220, as shown inFIG.2A. In some embodiments, the position sensors215-220may include one or more of magnetic sensing elements, such as Hall sensors, tunneling magnetoresistance (TMR) sensors, giant magnetoresistance (GMR) sensors, anisotropic magnetoresistance (AMR), etc. Position sensors215and220may each be represented by an equivalent circuit including resistance235a-250aand235b-250b, respectively. In some embodiments, the measurement circuitry200may include a plurality of position sensor magnets including a first position sensor magnet225and a second position sensor magnet230. The first and second position sensors215-220may be positioned proximate the first and second position sensor magnets225-230, respectively, such that the first position sensor215may measure predominantly the magnetic field produced by the first position sensor magnet225whilst the second position sensor220may measure predominantly the magnetic field produced by the first position sensor magnet230. The position sensors215and220may have terminals “Bias” and “GND” to receive the bias current from power supply255. With the exposure to the magnetic fields provided respectively by the position sensor magnets225-230, the position sensors215-220may produce respective measurement signals, such as output voltages v1and v2, across terminals “P” and “N”. Because the position sensors215-220measure the magnetic fields of different polarities, the voltages v1and v2may have opposite polarities, as described above inFIG.1A. In some embodiments, the position sensor magnets225-230may be arranged to provide the magnetic fields of different polarities detectable by the position sensors215-220. For instance, as shown inFIG.2A, the first position sensor215may detect predominantly the magnetic field from the south pole of the first position sensor magnet225, whilst the second position sensor220may detect predominantly the magnetic field from the north pole of the second position sensor magnet230. This way, the measurement signals v1and v2of the position sensors215and220may have opposite polarities. As shown inFIG.2A, in some embodiments, the position sensors215and220may be coupled in parallel in reverse directions to produce a combined sensor output vpn that may include an average of the respective sensor signals v1and v2. As shown inFIG.2A, the P terminal of the first position sensor215may be coupled to the N terminal of the second position sensor220, the N terminal of the first position sensor215may be coupled to the P terminal of the second position sensor220, and then the combined sensor output vpn may be provided from the respective coupled terminals. Combining the measurement signals v1and v2into one combined sensor output may result in a less number of analog signals, e.g., one analog signal vpn rather than two analog signals v1and v2, thus reducing exposure to signal processing noises as well as requirements on analog devices. As shown inFIG.2A, the combined sensor output vpn may be provided to signal processing circuit210. The signal processing circuit210may include an active front end (AFE)260to condition and amplify vpn, an A/D converter265to convert the analog signal vpn into a digital signal, and a processor270to determine the position for the optics assembly and/or image sensor of the camera based on the combined sensor output. As described above, once the measurement of the autofocus position and/or the image sensor position measurement become available, the camera system may use the position measurement as a feedback signal to control the movement of the optics assembly (e.g., in autofocus) and/or the image sensor (e.g., in OIS). For purposes of illustration, power supply255is shown to be part of the signal processing circuit210. In some embodiments, the power supply255may not have to be within the signal processing circuit210but instead be implemented by separate circuit(s) or device(s). The same may apply to other components shown inFIG.2A, such as the AFE260and/or A/D converter265.

FIG.2Bdepicts a schematic diagram of another example measurement circuitry for measuring the autofocus position and/or image sensor position of a camera system, according to some embodiments. In this example, actuator200bmay include a first and a second position sensor magnets225band230b, according to some embodiments. In some embodiments, the actuator200bmay include a first and a second position sensors215band22bb, respectively represented by an equivalent circuit comprising resistance235ab-250aband235bb-250bb. In some embodiments, the first and second position sensors215b-220bmay be positioned proximate the first and second position sensor magnets225b-230b, respectively, such that the first position sensor215bmay measure predominantly the magnetic field produced by the first position sensor magnet225bwhilst the second position sensor220bmay measure predominantly the magnetic field produced by the first position sensor magnet230b. In some embodiments, the position sensor magnets225b-230bmay be arranged to provide the magnetic fields with the same polarity detectable by the position sensors215b-220b. For instance, as shown inFIG.2B, the first position sensor215bmay detect predominantly the magnetic fluxes from north to south poles of the first position sensor magnet225b, and the second position sensor220bmay also detect substantially the magnetic fluxes from north to south poles of the second position sensor magnet230b. Accordingly, the measurement signals v1and v2of the position sensors215band220bmay have the same polarity. As shown inFIG.2B, in some embodiments, the position sensors215band220bmay be coupled in parallel to produce a combined sensor output vpn that may include an average of the respective sensor signals v1and v2. As shown inFIG.2B, the combined sensor output vpn may be provided to signal processing circuit210to determine the position information, e.g., the position, distance, and/or angle of the optics assembly and/or the image sensor of the camera, as described above.

FIG.3Aillustrates a top view of an example embodiment of an actuator of a camera system, according to some embodiments. In the example shown inFIG.3A, the actuator300may include an optics assembly305that may include one or more flat or curved optic lenses. The optics assembly305may be held in an optics holder310that also carries position sensor magnets (or probe magnets)315-320, e.g., to implement autofocus movements of the optics assembly305. A magnet holder325may hold position control magnets330-345and position sensors350-355, which may be any magnetic sensor or other applicable sensor, e.g., tunneling magnetoresistance (TMR) sensors. Simply speaking, a TMR sensor may be viewed as a thin-film component comprising a barrier layer sandwiched between two ferromagnetic layers (a free layer and a pin layer). The magnetization direction of the pin layer may have been fixed, whilst the magnetization direction of the free layer may change according to a triggering magnetic field, e.g., a magnetic field produced by a position sensor magnet. Accordingly, the electrical resistance of the TMR sensor may change along with this change in the free layer. For instance, the electrical resistance may reduce when the magnetization directions of the pin layer and free layer become parallel, causing the TMR sensor to become conductive. When the magnetization directions of the free and pin layers are antiparallel, the resistance may increase, causing the TMR sensor to become insulated. Because the electrical characteristics of the TMR sensor may vary with the triggering magnetic field, the TMR may be used to measure the triggering magnetic field, from which the position information of the corresponding magnet may thus be derived. For instance, the TMR sensor may be combined with one or more other resistors to form a voltage divider. With a bias current applied to the voltage divider including the TMR sensor, the sensor signal, e.g., a voltage across the P-N terminals of the TMR sensor, may be measured as an indication of the position, distance, and/or angle of a position sensor magnet with respect to the TMR sensor. Given the orientation-related characteristics of TMR sensors, the position sensors350-355may each have a sensitive direction, as indicated by the arrows inFIG.3A. When the directions of the magnetic fields from the position sensors315and320become parallel (or antiparallel) to the sensitive directions of the corresponding position sensors350-355, the position sensors350-355may behave as low-resistance (or high-resistance) components and produce small (or large) sensor signals, e.g., small (or large) voltages across respective P-N terminals. In some embodiments, the actuator300may further include position control magnets330-345. The position control magnets330-345may create magnetic fields, which the interior coils (including autofocus coil and/or optical image stabilization coil, not visible inFIG.3A) of the actuator module300may interfere with to produce motive forces (e.g., Lorentz forces) to move the optics assembly305or image sensor(s) of the camera system. As the position sensor magnets315-320are affixed to the optics holder310, the position sensor magnets315-320may move together with the optics assembly305relative to the position sensors350-355. The position sensor magnets315-320may provide magnetic fields, with which the position sensors350-355may interfere to produce sensor signals, e.g., output voltage signals, for measuring the position of the optics assembly305.

In some embodiments, the position sensor magnets315-320may provide magnetic fields with the same or different polarities detectable by the position sensors350and355, respectively. For instance, in the example shown inFIG.3A, the position sensor magnets315-320and position sensors350-355may be configured such that the sensitive directions of the position sensors315-320may become substantially parallel to the magnetic field directions (e.g., the directions of the magnetic fluxes from north to south poles) of the position sensor magnets315-320. As the position sensor magnets315-320move, e.g., with the optics holder310along the optical (Z) axis, the magnetic fields of the position sensor magnets315-320detectable by the position sensors350-355respectively may vary accordingly, and position information, e.g., the distance, position and/or angle, of the position sensor magnets315-320may thus be determined, e.g., relative to the position sensors350-355. In some embodiments, the position sensors350-355may be coupled, e.g., with direct or indirect electrical connections, in parallel in the same or reverse directions to produce a combined sensor output. For instance, as described above inFIG.2B, each of the two position sensors350-355may produce a sensor signal from P-N terminals. The two position sensors350-355may be coupled in parallel with the P terminal of the position sensor350coupled to the P terminal of the position sensor355and the N terminal of the position sensor350coupled to the N terminal of the position sensor355to produce a combined sensor output vpn. This connection may compensate for tilt effects in the sensitive directions of the position sensors350-355, e.g., the tile of the optics assembly causing different angles for the position sensor magnets315-320relative to the respective position sensors350-355. In some embodiments, the combined sensor output may include an average of the respective sensor signals from the first and second position sensors350-355. Further, as shown inFIG.3A, the position sensors350-355may be arranged spatially in substantially opposite directions such that the position sensors350-355may sense an external field with different polarities. For instance, in this example inFIG.3A, the external field direction may be antiparallel to the magnetic field direction of the position sensor magnet315(and sensitive direction of position sensor350) but parallel to the magnetic field direction of the position sensor magnet320(and sensitive direction of position sensor350). The external magnetic field may superposition upon the magnetic fields from the position sensor magnets315-320, such that the total magnetic field detectable by the position sensor350may be weakened whilst the total magnetic field seen by the other position sensor355may be strengthened. Thus, the sensor signal of position sensors350may be reduced and the sensor signal from position sensor355may be increased. In combination, the effect of the external magnetic field on the combined sensor output from the position sensors350-355may be canceled.

One with ordinary skills in the art shall appreciate that the example inFIG.3Ais merely an example for purposes of illustration. In some embodiments, the camera system may include less or more position sensor magnets315-320and less or more position sensors350-355. Further, when the camera system includes multiple position sensor magnets, such as position sensor magnets315-320, the position sensor magnets315-320may be positioned on the lens holder310in different manners. For instance, the position sensor magnets315-320may be placed at opposite locations around the optics assembly305, as shown inFIG.3A. Alternatively, the position sensor magnets315-320may be placed at the same side of the camera system. In some embodiments, the position sensor magnets315-320may be placed at diagonally opposite position, like the position control magnets330and340. As the arrangement of the position sensor magnets315-320changes, the placement of the position sensors350-355may be adjusted accordingly so that the position sensors350-355may still measure the magnetic fields of the position sensor magnets350-355respectively. InFIG.3, the position sensor magnets315-320are affixed to a moveable part such as the lens holder310of the camera system. Thus, the position sensor magnets315-320moves relative to stationary position sensors350-155. In some embodiments, the movability of the position sensor magnet(s) and position sensor(s) may be reversed. For instance, the position sensor magnets315-320may be installed on stationary magnet holder325, whilst the position sensors350-355affixed to the dynamic lens holder310. Besides the various ways to arrange the position sensor magnet(s) and position sensor(s), what is more important is that one component moves relative to the other component such that the varying magnetic field may be measured to determine the position of the moving component. In this example inFIG.3A, the camera system includes dedicated position sensor magnets315-320, separate from the position control magnets330-345. In some embodiments, the camera system may combine the functionalities of the position sensor magnets315-320into the position control magnets330-345, such as using the position control magnets330-345also as the probe magnets. Although the above various embodiments are described with respect to the position sensor magnet(s) and position sensor(s) for position measurement of the optics assembly305, the same applies to the image sensor(s) (not visible inFIG.3A) of the camera system as well.

FIG.3Bdepicts a top view of another example embodiment of an actuator of a camera system, according to some embodiments. In this example inFIG.3B, the actuator300bmay include an optics assembly305bthat may include one or more flat or curved optic lenses. The optics assembly305bmay be held in an optics holder310bthat also carries position sensor magnets (or probe magnets)315b,317b,319band320b, e.g., to implement autofocus movements of the optics assembly305b. A magnet holder325bmay hold position control magnets330b-345band position sensors350b,353b,354band355b, which may be any magnetic sensor or other applicable sensor, e.g., Hall sensors, tunneling magnetoresistance (TMR) sensors, giant magnetoresistance (GMR) sensors, anisotropic magnetoresistance (AMR), and so on. The position control magnets330b-345bmay create magnetic fields, which the interior coils (not shown, including autofocus coil and/or optical image stabilization coil) of the actuator module300bmay interfere with to produce motive forces (e.g., Lorentz forces) to move the optics assembly305bor image sensor(s) of the camera system. As the position sensor magnets315b-320bare affixed to the optics holder310b, the position sensor magnets315b-320bmay move together with the optics assembly305brelative to the position sensors350b-355b. The position sensor magnets315b-320bmay provide magnetic fields, with which the position sensors350b-355bmay interfere to produce sensor signals, e.g., output voltage signals, for measuring the position of the optics assembly305b. As described above, the position sensor magnets315b-320bas well as the position sensors350b-355bmay be arranged in various configurations to provide the position measurement for the optics assembly305band/or the image sensor(s) of the camera system. For instance, the position sensor magnets315band317bmay be arranged to provide magnetic fields of different polarities detectable by position sensors350b-353b, whilst the319band320bmay be arranged to provide magnetic fields of different polarities detectable by position sensors355b-355b, respectively. The position sensors350b-353bmay be coupled in parallel in reverse directions to produce a combined sensor output for determining the position for the optics assembly305bon the left side of the camera system, and the position sensors354b-355bto produce a combined sensor output for determining the position for the optics assembly305bon the right side of the camera system. In some embodiments, it may be the position sensor magnets315band320bthat are arranged to provide different-polarity magnetic fields so that the position sensors350band355bare coupled in parallel in reverse direction. In some embodiments, all of the four position sensors350b-355bmay be coupled in parallel in the same or reverse directions to produce the combined sensor output for position measurement. Again, with multiple position sensor magnets and position sensors, the components may be arranged in various ways. As shown inFIGS.1-3, a camera or an actuator of a camera may have position sensor magnets and position sensors in various configurations. For example, some position sensors may be arranged spatially in a substantial same direction, the position sensors may detect the magnetic fields from corresponding position sensor magnets with different polarities, and the position sensors may be coupled in parallel in reverse directions to produce a combined sensor output (and cancel the effect of an external magnetic field), according to some embodiments, as described above inFIGS.1A,2A. In some embodiments, some position sensors may be arranged spatially in substantial opposite directions, the position sensors may detect the magnetic fields from corresponding position sensor magnets with the same polarity, and the position sensors may be coupled in parallel in the same direction to produce a combined sensor output (and cancel the effect of an external magnetic field), according to some embodiments, as described above inFIGS.1B,2B and3A. Further, regardless of the configurations of the position sensor magnets and position sensors, the combined sensor output may be created in analog circuits, which would reduce the burden and requirement on the digital circuits and associated signal processing, as described above. Note thatFIGS.1-3are merely examples for purposes of illustration. The camera or the actuator of the camera may employ different kinds of position sensor(s) and position sensor magnet(s) in the same or different configurations, as described above, for measuring different position information, e.g., the position information of the optics assembly (e.g., in autofocus) and the image sensor (e.g., in OIS).

FIG.4depicts a side view of an example embodiment of an actuator of a camera system, according to some embodiments. As shown inFIG.4, an actuator package400may include a base assembly or substrate408, an optics assembly402, and a cover412. Base assembly408may include one or more of, but is not limited to, a base4008, supporting one or more position sensors (e.g., Hall sensors)410and suspension wires420, which enable magnetic sensing for autofocus position detection by detecting movements of position sensor magnets418. In some embodiments, there are four suspension wires420. An optics assembly402may be suspended on the base assembly408by suspension of the upper springs440on the suspension wires420. The actuator400may include one or more of, but is not limited to, optics assembly402, optics holder (autofocus coil)404, magnet(s)406, upper spring(s)430, and lower spring(s)442. The upper and lower spring(s) may be collectively referred to herein as optics springs. In optics assembly402, one or more optics components (e.g., a lens or lens assembly) may be screwed, mounted or otherwise held in or by an optics holder (autofocus coil)404. In some embodiments, the optics assembly402/optics holder (autofocus coil) assembly404may be suspended from or attached to the position control magnets406by upper spring(s)440, and lower spring(s)442, and the position control magnets406may be rigidly mounted to base408. The upper spring(s)440and lower spring(s)442may be flexible to allow the optics assembly404a range of motion along the Z (optical) axis for optical focusing, and wires420may be flexible to allow a range of motion on the XY plane orthogonal to the Z (optical) axis for optical image stabilization. In addition to the position sensor magnets418an actuator module400as described herein may include one or more position sensor magnets418and one or more position control magnets406. In some embodiments, a package of processors and memory490or other computer-readable medium may, in some embodiments, be included in actuator module400. In some embodiments, a package of processors and memory490or other computer-readable medium as described herein may alternatively, in some embodiments, be omitted from actuator module400and housed elsewhere in a device in which actuator package400is installed. As described above, the measurement of the position for the optics assembly402and/or image sensor450may be merely one of the early steps in control of the movements of the optics assembly402(e.g., in autofocus movements) and/or image sensor450(e.g., in OIS movements). Once the position measurement becomes available, the camera system may take the position measurement as a feedback signal and drive the actuator400to move and control the positions of the optics assembly402and/or the image sensor450based on the position measurement feedback. Note that the embodiments illustrated inFIG.4is merely one example for purposes of illustration. As described above, the camera system may include more or less position sensor magnets and more or less position sensors for measuring the positions of the optics assembly402and/or the image sensor450. Further, as described above, the position sensor magnets and position sensors may be arranged in various configurations. For instance, in some embodiments, the position sensor magnets may be configured to be moveable (with the optics assembly402and/or the image sensor450) while the position sensors are stationary (with respect to the base408). In some other embodiments, the position sensor magnets may be stationary (with respect to the base408) whilst the position sensors are moveable (with the optics assembly402and/or the image sensor450). In some embodiments, the position sensor magnets may be arranged to produce magnetic fields of different polarities detectable by the position sensors. The position sensors may be coupled in parallel in reverse directions to produce a combined sensor output. The camera system may use the combined sensor output to determine the position for the optics assembly402and/or the image sensor450. In some embodiments, the position sensor magnets (and position sensors) may be positioned at or near the same locations or at different places (e.g., diagonally opposite places) around the camera system. InFIG.4, the optics assembly402may be controllably moveable along the Z axis (i.e., autofocus movements) as well as on the XY plane (i.e., OIS movements). In some embodiments, the optics assembly402may be controlled to move along the Z axis (i.e., autofocus movements), and the image sensor450may be controlled to shift on the XY plane (i.e., OIS movements). For instance, the image sensor450may be affixed to a dynamic platform which may be further suspended from the base408or other stationary parts of the actuator400(e.g., the cover412) through mechanical flexures, e.g., in the horizontal directions on the XY plane that is orthogonal to the Z axis (i.e., the optical axis) of the optics assembly402. The flexure may limit but also allow for certain degrees of freedom for the image sensor to move relative to the optics assembly402on the XY plane.

FIG.5Adepicts a plot of a relationship between position sensor voltage and magnet position, according to some embodiments. Graph502includes a voltage-position curve512indicating that position sensor voltage522changes as a function of magnet position532, relative to the sensor. Take the Hall position sensor as an example. The sensor may conduct a constant bias current while exposing to a magnetic field. The magnetic field may create Lorentz forces on the electrons moving through the sensor, such that a difference in electric potential—an output voltage—may develop between the two sides of the sensor. Note that for a range of positions, the position sensor output is approximately linear with magnet position532.

FIG.5Billustrates a schematic view of a position sensor interacting with a magnet to provide magnetic sensing, according to some embodiments. A magnetic sensing element560(e.g., a Hall sensor) on a substrate570may generate a measurement of a magnetic field resulting from the position sensor magnet (or probe magnet)520. In this example, the position sensor520may be fixedly mounted to the camera lens carrier530and thus become moveable with the camera lens carrier530, e.g., in autofocus movements. When the position sensor magnet520moves from a closer position Z1540to a further position Z2550, with respect to the magnetic sensing element560, the magnetic sensing element560may produce respective sensor signals, e.g., the position sensor voltages V1and V2, as shown inFIG.5A. Because the position sensor520moves away from the magnetic sensing element560, the position sensor voltage becomes smaller, (i.e., v2<v1).

FIG.6depicts a flowchart showing the autofocus and/or image sensor position control based on position measurement, according to some embodiments. As indicated at605, a camera system, e.g., the camera system described above inFIGS.1-5, may provide a plurality of position sensors including a first and a second position sensors, arranged in various (e.g., spatial) configurations, to measure the magnetic fields of different polarities. As described above, e.g., inFIG.1A-1B, the first and second position sensors130-135(or130-135) may produce sensor signals v1and v2corresponding to the measurement of respective magnetic fields produced by the position sensor magnets120-125(or120b-125b), which have different or same polarities. As indicated at610, the position sensors may be coupled in parallel in reverse or same directions to produce a combined sensor output. For instance, as described above inFIG.1A-1B, the P terminal of the first position sensor130(or130b) may be coupled to the N (or P) terminal of the second position sensor135(or135b) whilst the N terminal of the first position sensor130(or130b) may be coupled to the P (or N) terminal of the second position sensor135(or135b) to produce the combined sensor output vpn. As indicated at615, the camera system may determine position information for the optics assembly and/or image sensor, e.g., the autofocus position (e.g., the position of the optics assembly) and/or the image sensor position, based in part on the combined sensor output. As described above inFIG.1A-1B, the camera system may provide the combined sensor output vpn to the signal processing circuit510to determine the autofocus position and/or the image sensor position. As indicated at620, once the position information, e.g., the autofocus position and/or the image sensor position, becomes available, the camera system may use the position measurement as a feedback signal to control the position of the optics assembly (e.g., in autofocus) and/or the position of the image sensor (e.g., in OIS).

Attention is now directed toward embodiments of portable devices with cameras.FIG.7illustrates a block diagram of an example portable multifunction device700that may include a camera system as described above, according to some embodiments. Cameras764are sometimes called “optical sensors” for convenience, and may also be known as or called an optical sensor system. Device700may include memory702(which may include one or more computer readable storage mediums), memory controller722, one or more processing units (CPUs)720, peripherals interface718, RF circuitry708, audio circuitry710, speaker711, touch-sensitive display system712, microphone713, input/output (I/O) subsystem706, other input or control devices716, and external port724. Device700may include multiple. optical sensors764. These components may communicate over one or more communication buses or signal lines703.

It should be appreciated that device700is only one example of a portable multifunction device, and that device700may have more or fewer components than shown, may combine two or more components, or may have a different configuration or arrangement of the components. The various components shown inFIG.7may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

Memory702may include high-speed random access memory and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state memory devices. Access to memory702by other components of device700, such as CPU720and the peripherals interface718, may be controlled by memory controller722.

Peripherals interface718can be used to couple input and output peripherals of the device to CPU720and memory702. The one or more processors720run or execute various software programs and/or sets of instructions stored in memory702to perform various functions for device700and to process data.

In some embodiments, peripherals interface718, CPU720, and memory controller722may be implemented on a single chip, such as chip704. In some other embodiments, they may be implemented on separate chips.

Audio circuitry710, speaker711, and microphone713provide an audio interface between a user and device700. Audio circuitry710receives audio data from peripherals interface718, converts the audio data to an electrical signal, and transmits the electrical signal to speaker711. Speaker711converts the electrical signal to human-audible sound waves. Audio circuitry710also receives electrical signals converted by microphone713from sound waves. Audio circuitry710converts the electrical signal to audio data and transmits the audio data to peripherals interface718for processing. Audio data may be retrieved from and/or transmitted to memory702and/or RF circuitry708by peripherals interface718. In some embodiments, audio circuitry710also includes a headset jack (e.g.,812,FIG.8). The headset jack provides an interface between audio circuitry710and removable audio input/output peripherals, such as output-only headphones or a headset with both output (e.g., a headphone for one or both ears) and input (e.g., a microphone).

I/O subsystem706couples input/output peripherals on device700, such as touch screen712and other input control devices716, to peripherals interface718. I/O subsystem706may include display controller756and one or more input controllers760for other input or control devices. The one or more input controllers760receive/send electrical signals from/to other input or control devices716. The other input control devices716may include physical buttons (e.g., push buttons, rocker buttons, etc.), dials, slider switches, joysticks, click wheels, and so forth. In some alternate embodiments, input controller(s)760may be coupled to any (or none) of the following: a keyboard, infrared port, USB port, and a pointer device such as a mouse. The one or more buttons (e.g.,808,FIG.8) may include an up/down button for volume control of speaker711and/or microphone713. The one or more buttons may include a push button (e.g.,806,FIG.8).

Touch-sensitive display712provides an input interface and an output interface between the device and a user. Display controller756receives and/or sends electrical signals from/to touch screen712. Touch screen712displays visual output to the user. The visual output may include graphics, text, icons, video, and any combination thereof (collectively termed “graphics”). In some embodiments, some or all of the visual output may correspond to user-interface objects.

Touch screen712has a touch-sensitive surface, sensor or set of sensors that accepts input from the user based on haptic and/or tactile contact. Touch screen712and display controller756(along with any associated modules and/or sets of instructions in memory702) detect contact (and any movement or breaking of the contact) on touch screen712and converts the detected contact into interaction with user-interface objects (e.g., one or more soft keys, icons, web pages or images) that are displayed on touch screen712. In an example embodiment, a point of contact between touch screen712and the user corresponds to a finger of the user.

Touch screen712may have a video resolution in excess of 700 dpi. In some embodiments, the touch screen has a video resolution of approximately 760 dpi. The user may make contact with touch screen712using any suitable object or appendage, such as a stylus, a finger, and so forth. In some embodiments, the user interface is designed to work primarily with finger-based contacts and gestures, which can be less precise than stylus-based input due to the larger area of contact of a finger on the touch screen. In some embodiments, the device translates the rough finger-based input into a precise pointer/cursor position or command for performing the actions desired by the user.

Device700also includes power system762for powering the various components. Power system762may include a power management system, one or more power sources (e.g., battery, alternating current (AC)), a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator (e.g., a light-emitting diode (LED)) and any other components associated with the generation, management and distribution of power in portable devices.

Device700may also include one or more optical sensors or cameras764.FIG.7shows an optical sensor764coupled to optical sensor controller758in I/O subsystem706. Optical sensor764may include charge-coupled device (CCD) or complementary metal-oxide semiconductor (CMOS) phototransistors. Optical sensor764receives light from the environment, projected through one or more lens, and converts the light to data representing an image. In conjunction with imaging module743(also called a camera module), optical sensor764may capture still images or video. In some embodiments, an optical sensor764is located on the back of device700, opposite touch screen display712on the front of the device, so that the touch screen display712may be used as a viewfinder for still and/or video image acquisition. In some embodiments, another optical sensor is located on the front of the device so that the user's image may be obtained for videoconferencing while the user views the other video conference participants on the touch screen display.

Device700may also include one or more proximity sensors766.FIG.7shows proximity sensor766coupled to peripherals interface718. Alternately, proximity sensor766may be coupled to input controller760in I/O subsystem706. In some embodiments, the proximity sensor766turns off and disables touch screen712when the multifunction device700is placed near the user's ear (e.g., when the user is making a phone call).

Device700includes one or more orientation sensors768. In some embodiments, the one or more orientation sensors768include one or more accelerometers (e.g., one or more linear accelerometers and/or one or more rotational accelerometers). In some embodiments, the one or more orientation sensors768include one or more gyroscopes. In some embodiments, the one or more orientation sensors768include one or more magnetometers. In some embodiments, the one or more orientation sensors768include one or more of global positioning system (GPS), Global Navigation Satellite System (GLONASS), and/or other global navigation system receivers. The GPS, GLONASS, and/or other global navigation system receivers may be used for obtaining information concerning the location and orientation (e.g., portrait or landscape) of device700. In some embodiments, the one or more orientation sensors768include any combination of orientation/rotation sensors.FIG.7shows the one or more orientation sensors768coupled to peripherals interface718. Alternately, the one or more orientation sensors768may be coupled to an input controller760in I/O subsystem706. In some embodiments, information is displayed on the touch screen display712in a portrait view or a landscape view based on an analysis of data received from the one or more orientation sensors768.

In some embodiments, the software components stored in memory702include operating system726, communication module (or set of instructions)728, contact/motion module (or set of instructions)730, graphics module (or set of instructions)732, text input module (or set of instructions)734, Global Positioning System (GPS) module (or set of instructions)735, arbiter module758and applications (or sets of instructions)736. Furthermore, in some embodiments memory702stores device/global internal state757. Device/global internal state757includes one or more of: active application state, indicating which applications, if any, are currently active; display state, indicating what applications, views or other information occupy various regions of touch screen display712; sensor state, including information obtained from the device's various sensors and input control devices716; and location information concerning the device's location and/or attitude.

Communication module728facilitates communication with other devices over one or more external ports724and also includes various software components for handling data received by RF circuitry708and/or external port724. External port724(e.g., Universal Serial Bus (USB), FIREWIRE, etc.) is adapted for coupling directly to other devices or indirectly over a network (e.g., the Internet, wireless LAN, etc.). In some embodiments, the external port is a multi-pin (e.g., 30-pin) connector.

Contact/motion module730may detect contact with touch screen712(in conjunction with display controller756) and other touch sensitive devices (e.g., a touchpad or physical click wheel). Contact/motion module730includes various software components for performing various operations related to detection of contact, such as determining if contact has occurred (e.g., detecting a finger-down event), determining if there is movement of the contact and tracking the movement across the touch-sensitive surface (e.g., detecting one or more finger-dragging events), and determining if the contact has ceased (e.g., detecting a finger-up event or a break in contact). Contact/motion module730receives contact data from the touch-sensitive surface. Determining movement of the point of contact, which is represented by a series of contact data, may include determining speed (magnitude), velocity (magnitude and direction), and/or an acceleration (a change in magnitude and/or direction) of the point of contact. These operations may be applied to single contacts (e.g., one finger contacts) or to multiple simultaneous contacts (e.g., “multitouch”/multiple finger contacts). In some embodiments, contact/motion module730and display controller756detect contact on a touchpad.

Contact/motion module730may detect a gesture input by a user. Different gestures on the touch-sensitive surface have different contact patterns. Thus, a gesture may be detected by detecting a particular contact pattern. For example, detecting a finger tap gesture includes detecting a finger-down event followed by detecting a finger-up (lift off) event at the same position (or substantially the same position) as the finger-down event (e.g., at the position of an icon). As another example, detecting a finger swipe gesture on the touch-sensitive surface includes detecting a finger-down event followed by detecting one or more finger-dragging events, and subsequently followed by detecting a finger-up (lift off) event.

Graphics module732includes various known software components for rendering and displaying graphics on touch screen712or other display, including components for changing the intensity of graphics that are displayed. As used herein, the term “graphics” includes any object that can be displayed to a user, including without limitation text, web pages, icons (such as user-interface objects including soft keys), digital images, videos, animations and the like.

In some embodiments, graphics module732stores data representing graphics to be used. Each graphic may be assigned a corresponding code. Graphics module732receives, from applications etc., one or more codes specifying graphics to be displayed along with, if necessary, coordinate data and other graphic property data, and then generates screen image data to output to display controller756.

Text input module734, which may be a component of graphics module732, provides soft keyboards for entering text in various applications (e.g., contacts737, e-mail740, IM741, browser747, and any other application that needs text input).

GPS module735determines the location of the device and provides this information for use in various applications (e.g., to telephone738for use in location-based dialing, to camera743as picture/video metadata, and to applications that provide location-based services such as weather widgets, local yellow page widgets, and map/navigation widgets).

Applications736may include the following modules (or sets of instructions), or a subset or superset thereof:contacts module737(sometimes called an address book or contact list);telephone module738;video conferencing module739;e-mail client module740;instant messaging (IM) module741;workout support module742;camera module743for still and/or video images;image management module744;browser module747;calendar module748;widget modules749, which may include one or more of: weather widget749-1, stocks widget749-2, calculator widget749-3, alarm clock widget749-4, dictionary widget749-5, and other widgets obtained by the user, as well as user-created widgets749-6;widget creator module750for making user-created widgets749-6;search module751;video and music player module752, which may be made up of a video player module and a music player module;notes module753;map module754; and/oronline video module755.

Examples of other applications736that may be stored in memory702include other word processing applications, other image editing applications, drawing applications, presentation applications, JAVA-enabled applications, encryption, digital rights management, voice recognition, and voice replication.

In conjunction with touch screen712, display controller756, contact module730, graphics module732, and text input module734, contacts module737may be used to manage an address book or contact list (e.g., stored in application internal state757), including: adding name(s) to the address book; deleting name(s) from the address book; associating telephone number(s), e-mail address(es), physical address(es) or other information with a name; associating an image with a name; categorizing and sorting names; providing telephone numbers or e-mail addresses to initiate and/or facilitate communications by telephone738, video conference739, e-mail740, or IM741; and so forth.

In conjunction with RF circuitry708, audio circuitry710, speaker711, microphone713, touch screen712, display controller756, contact module730, graphics module732, and text input module734, telephone module738may be used to enter a sequence of characters corresponding to a telephone number, access one or more telephone numbers in address book737, modify a telephone number that has been entered, dial a respective telephone number, conduct a conversation and disconnect or hang up when the conversation is completed. As noted above, the wireless communication may use any of a variety of communications standards, protocols and technologies.

In conjunction with RF circuitry708, audio circuitry710, speaker711, microphone713, touch screen712, display controller756, optical sensor764, optical sensor controller758, contact module730, graphics module732, text input module734, contact list737, and telephone module738, videoconferencing module739includes executable instructions to initiate, conduct, and terminate a video conference between a user and one or more other participants in accordance with user instructions.

In conjunction with RF circuitry708, touch screen712, display controller756, contact module730, graphics module732, and text input module734, e-mail client module740includes executable instructions to create, send, receive, and manage e-mail in response to user instructions. In conjunction with image management module744, e-mail client module740makes it very easy to create and send e-mails with still or video images taken with camera module743.

In conjunction with RF circuitry708, touch screen712, display controller756, contact module730, graphics module732, text input module734, GPS module735, map module754, and music player module746, workout support module742includes executable instructions to create workouts (e.g., with time, distance, and/or calorie burning goals); communicate with workout sensors (sports devices); receive workout sensor data; calibrate sensors used to monitor a workout; select and play music for a workout; and display, store and transmit workout data.

In conjunction with touch screen712, display controller756, optical sensor(s)764, optical sensor controller758, contact module730, graphics module732, and image management module744, camera module743includes executable instructions to capture still images or video (including a video stream) and store them into memory702, modify characteristics of a still image or video, or delete a still image or video from memory702.

In conjunction with touch screen712, display controller756, contact module730, graphics module732, text input module734, and camera module743, image management module744includes executable instructions to arrange, modify (e.g., edit), or otherwise manipulate, label, delete, present (e.g., in a digital slide show or album), and store still and/or video images.

In conjunction with RF circuitry708, touch screen712, display system controller756, contact module730, graphics module732, and text input module734, browser module747includes executable instructions to browse the Internet in accordance with user instructions, including searching, linking to, receiving, and displaying web pages or portions thereof, as well as attachments and other files linked to web pages.

In conjunction with RF circuitry708, touch screen712, display system controller756, contact module730, graphics module732, text input module734, e-mail client module740, and browser module747, calendar module748includes executable instructions to create, display, modify, and store calendars and data associated with calendars (e.g., calendar entries, to do lists, etc.) in accordance with user instructions.

In conjunction with RF circuitry708, touch screen712, display system controller756, contact module730, graphics module732, text input module734, and browser module747, widget modules749are mini-applications that may be downloaded and used by a user (e.g., weather widget749-1, stocks widget749-2, calculator widget749-3, alarm clock widget749-4, and dictionary widget749-5) or created by the user (e.g., user-created widget749-6). In some embodiments, a widget includes an HTML (Hypertext Markup Language) file, a CSS (Cascading Style Sheets) file, and a JavaScript file. In some embodiments, a widget includes an XML (Extensible Markup Language) file and a JavaScript file (e.g., Yahoo! Widgets).

In conjunction with RF circuitry708, touch screen712, display system controller756, contact module730, graphics module732, text input module734, and browser module747, the widget creator module750may be used by a user to create widgets (e.g., turning a user-specified portion of a web page into a widget).

In conjunction with touch screen712, display system controller756, contact module730, graphics module732, and text input module734, search module751includes executable instructions to search for text, music, sound, image, video, and/or other files in memory702that match one or more search criteria (e.g., one or more user-specified search terms) in accordance with user instructions.

In conjunction with touch screen712, display system controller756, contact module730, graphics module732, audio circuitry710, speaker711, RF circuitry708, and browser module747, video and music player module752includes executable instructions that allow the user to download and play back recorded music and other sound files stored in one or more file formats, such as MP3 or AAC files, and executable instructions to display, present or otherwise play back videos (e.g., on touch screen712or on an external, connected display via external port724). In some embodiments, device700may include the functionality of an MP3 player.

In conjunction with touch screen712, display controller756, contact module730, graphics module732, and text input module734, notes module753includes executable instructions to create and manage notes, to do lists, and the like in accordance with user instructions.

In conjunction with RF circuitry708, touch screen712, display system controller756, contact module730, graphics module732, text input module734, GPS module735, and browser module747, map module754may be used to receive, display, modify, and store maps and data associated with maps (e.g., driving directions; data on stores and other points of interest at or near a particular location; and other location-based data) in accordance with user instructions.

In conjunction with touch screen712, display system controller756, contact module730, graphics module732, audio circuitry710, speaker711, RF circuitry708, text input module734, e-mail client module740, and browser module747, online video module755includes instructions that allow the user to access, browse, receive (e.g., by streaming and/or download), play back (e.g., on the touch screen or on an external, connected display via external port724), send an e-mail with a link to a particular online video, and otherwise manage online videos in one or more file formats, such as H.264. In some embodiments, instant messaging module741, rather than e-mail client module740, is used to send a link to a particular online video.

In some embodiments, device700is a device where operation of a predefined set of functions on the device is performed exclusively through a touch screen and/or a touchpad. By using a touch screen and/or a touchpad as the primary input control device for operation of device700, the number of physical input control devices (such as push buttons, dials, and the like) on device700may be reduced.

FIG.8depicts illustrates an example portable multifunction device700that may include a camera system as described above, according to some embodiments. The device700may have a touch screen712. The touch screen712may display one or more graphics within user interface (UI)800. In this embodiment, as well as others described below, a user may select one or more of the graphics by making a gesture on the graphics, for example, with one or more fingers802. (not drawn to scale in the figure) or one or more styluses803(not drawn to scale in the figure).

Device700may also include one or more physical buttons, such as “home” or menu button804. As described previously, menu button804may be used to navigate to any application736in a set of applications that may be executed on device700. Alternatively, in some embodiments, the menu button804is implemented as a soft key in a GUI displayed on touch screen712.

In one embodiment, device700includes touch screen712, menu button804, push button806for powering the device on/off and locking the device, volume adjustment button(s)808, Subscriber Identity Module (SIM) card slot810, head set jack812, and docking/charging external port824. Push button806may be used to turn the power on/off on the device by depressing the button and holding the button in the depressed state for a predefined time interval; to lock the device by depressing the button and releasing the button before the predefined time interval has elapsed; and/or to unlock the device or initiate an unlock process. In an alternative embodiment, device700also may accept verbal input for activation or deactivation of some functions through microphone713.

It should be noted that, although many of the examples herein are given with reference to optical sensor(s)/camera(s)764(on the front of a device), one or more rear-facing cameras or optical sensors that are pointed opposite from the display may be used instead of, or in addition to, an optical sensor(s)/camera(s)764on the front of a device.

Example Computer System

FIG.9illustrates an example computing device, referred to as computer system900, that may include or host embodiments of a camera system as described above with reference toFIGS.1-8. In addition, computer system900may implement methods for controlling operations of the camera and/or for performing image processing images captured with the camera.

In the illustrated embodiment, computer system900includes one or more processors902coupled to a system memory904via an input/output (I/O) interface906. Computer system900further includes one or more cameras908coupled to the I/O interface906. Computer system900further includes a network interface910coupled to I/O interface906, and one or more input/output devices912, such as cursor control device914, keyboard916, and display(s)918. In some cases, it is contemplated that embodiments may be implemented using a single instance of computer system900, while in other embodiments multiple such systems, or multiple nodes making up computer system900, may be configured to host different portions or instances of embodiments. For example, in one embodiment some elements may be implemented via one or more nodes of computer system900that are distinct from those nodes implementing other elements.

In various embodiments, computer system900may be a uniprocessor system including one processor902, or a multiprocessor system including several processors902(e.g., two, four, eight, or another suitable number). Processors902may be any suitable processor capable of executing instructions. For example, in various embodiments processors902may be general-purpose or embedded processors implementing any of a variety of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, or MIPS ISAs, or any other suitable ISA. In multiprocessor systems, each of processors902may commonly, but not necessarily, implement the same ISA.

System memory904may be configured to store program instructions920accessible by processor902. In various embodiments, system memory904may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), nonvolatile/Flash-type memory, or any other type of memory. Additionally, existing camera control data922of memory904may include any of the information or data structures described above. In some embodiments, program instructions920and/or data922may be received, sent or stored upon different types of computer-accessible media or on similar media separate from system memory904or computer system900. In various embodiments, some or all of the functionality described herein may be implemented via such a computer system900.

In one embodiment, I/O interface906may be configured to coordinate I/O traffic between processor902, system memory904, and any peripheral devices in the device, including network interface910or other peripheral interfaces, such as input/output devices912. In some embodiments, I/O interface906may perform any necessary protocol, timing or other data transformations to convert data signals from one component (e.g., system memory904) into a format suitable for use by another component (e.g., processor902). In some embodiments, I/O interface906may include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard, for example. In some embodiments, the function of I/O interface906may be split into two or more separate components, such as a north bridge and a south bridge, for example. Also, in some embodiments some or all of the functionality of I/O interface906, such as an interface to system memory904, may be incorporated directly into processor902.

Input/output devices912may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other devices suitable for entering or accessing data by one or more computer systems900. Multiple input/output devices912may be present in computer system900or may be distributed on various nodes of computer system900. In some embodiments, similar input/output devices may be separate from computer system900and may interact with one or more nodes of computer system900through a wired or wireless connection, such as over network interface910.

The various systems and methods as illustrated in the figures and described herein represent example embodiments of methods. The systems and methods may be implemented manually, in software, in hardware, or in a combination thereof. The order of any method may be changed, and various elements may be added, reordered, combined, omitted, modified, etc.

Although the embodiments above have been described in considerable detail, numerous variations and modifications may be made as would become apparent to those skilled in the art once the above disclosure is fully appreciated. It is intended that the following claims be interpreted to embrace all such modifications and changes and, accordingly.