Detection API

Introduction

The Detection API returns the location of any people and faces found in an image. Faces can be analyzed for gender, age, pose, or emotion; and a landmark detector can find the various facial features in the detected faces, including eyes, nose and mouth, by fitting 68 landmark points to those features.

Before proceeding, please ensure you have created a Sighthound Cloud API Token. Code samples for a variety of languages are available on the Sighthound Cloud demo page.

Usage Notes

API Base URL

Select the appropriate base URL for your environment and Cloud API account level. All Cloud accounts can use the free, development service for up to 5,000 API calls per month. Paid accounts (Basic, Pro, and Enterprise) have access to separate, production-ready servers.

Development: https://dev.sighthoundapi.com
Production (Paid Accounts): https://prod.sighthoundapi.com

General

Blurry scenes (possibly caused by camera settings or weather conditions) will reduce the accuracy of the object detector. The blurrier the object the more difficult to detect.
Good lighting conditions are important for detection. Detections may not trigger if the scene is too dark.
Angle of the camera to the object being detected should be within 45 degrees for best results.

Person Detection

The height of the person in the image should be at least 15% of image height.
For detecting people, the head and shoulders of the person are important. In cases where the head and shoulders of the person are occluded, detection might not be possible.
Unusual poses (e.g. crawling, lying down) are currently not handled. The system can be trained to handle such poses in the future.

Face Detection

The size of the face in the image should be at least 40x40 pixels.
The detection will handle faces occluded up to approximately 50%, given that the eyes are not occluded. Eyes are an essential for reliable detection.
The system is able to handle frontal with a high degree of accuracy. Profile and rotated faces (maximum of 45 degree, both in-plane and out-of-plane) are handled but are less accurate.
Wearable objects (e.g. glasses, cap, hat) should not cause as problems as long as more than 50% of the face is visible (including the eyes).
Scores greater than 0.5 are fairly confident for attributes returned with confidence scores (e.g. age and gender).

Detections ¶

Detect Objects ¶

Notes

Supported image formats are JPEG, GIF, and PNG.
Images larger than 1920x1080 will be scaled to a size within those bounds while respecting aspect ratio.
The minimum size supported for detection of a face in the image is 40x40 pixels. For better results, use faces sized 256x256 pixels or higher.

Using Image URL
POST`/v1/detections?type={type}&faceOption={faceOption}`

Example URI

POST https://dev.sighthoundapi.com/v1/detections?type=face,person&faceOption=gender,landmark,age,emotion,pose

URI Parameters

HideShow

type: string (optional) Example: face,person
Comma separated list of object categories to detect. Valid options are ‘all’, ‘face’, ‘person’. Default is ‘all’.
faceOption: string (optional) Example: gender,landmark,age,emotion,pose
For type ‘face’, additional detections can be performed by passing comma-separated list of values. Valid options are ‘gender’, ‘landmark’, ‘emotion’, ‘pose’, and ‘age’. Default returns face bounding box only.

Request

Request Body Attributes (JSON)

image (string): The image to analyze. This can be either a URL to an image (authentication data in URL is accepted) or inline as base64 encoded data. The maximum accepted image size is 10Mb.

Headers

Content-Type: application/json
X-Access-Token: developersOwnCloudAccessToken

Body

{
  "image": "http://www.example.com/path/to/image.jpg"
}

Response 200

Successful queries will return a 200 response with a JSON body describing all detected objects and the attributes of the processed image.

Response Attributes (JSON)

image (object): Information about the processed image.
- width (integer): Pixel width of source image
- height (integer): Pixel height of source image
- orientation (integer): EXIF code representing rotation and/or flipping of source Image needed to align the source photo with the detection coordinates. Only applicable for photos with EXIF data (e.g., photos taken on a mobile device). Defaults to 1 (no rotation or flip), else the value found in the image’s EXIF data and indicates that bounding vertices have been translated to match that coordinate space.
objects (array): An array of all detected objects. Each object contains:
- type (string): The label applied to the object, face or person.
- boundingBox- (object): An object containing x, y, width, and height values defining the location of the object in the image. The top left pixel of the image represents coordinate (0,0).
- attributes (object): describes attributes of the detected Object
  - gender (string): The computed gender of the face. Always returns a value of either male or female. Returned only for objects with type=face.
  - genderConfidence (float): A normalized value between 0 - 1 that represents the confidence that the computed gender is correct.
  - age (integer): The estimated age of the face. Always returns a value from 0 to 100. Returned only for objects with type=face.
  - ageConfidence (float): A normalized value between 0 - 1 that represents the confidence of the estimated age.
  - emotion (string): The top computed emotion of the face. Returned only for objects with type=face.
  - emotionConfidence (float): A normalized value between 0 - 1 that represents the confidence of the top computed emotion. Returned only for objects with type=face.
  - emotionsAll (object): All computed emotions. Returned only for objects with type=face.
  - pose (object): The pitch, roll, and yaw of the detected pose. Returned only for objects with type=face.
  - frontal (string): Frontal is true if the face is looking at camera and false if it’s a profile view. Returned only for objects with type=face.
- landmarks (object): Contains x/y coordinates for several parts of the detected face. See the Sighthound Cloud Demo for placement.

Headers

Content-Type: application/json

Body

{
    "image": {
        "width": 1280, 
        "height": 960, 
        "orientation": 1
    },
    "objects": [
        {
            "type": "person",
            "boundingBox": { 
                "x": 363, 
                "y": 182, 
                "height": 778, 
                "width": 723
            } 
        },
        {
            "type": "face",
            "boundingBox": {
                "x": 508, 
                "y": 305, 
                "height": 406, 
                "width": 406
            },
            "attributes": {
                "gender": "male", 
                "genderConfidence": 0.9883, 
                "age":25,
                "ageConfidence": 0.2599,
                "emotion": "happiness",
                "emotionConfidence": 0.9943,
                "emotionsAll": {
                    "neutral": 0.0018,
                    "sadness": 0.0009,
                    "disgust": 0.0002,
                    "anger": 0.0003,
                    "surprise": 0,
                    "fear": 0.0022,
                    "happiness": 0.9943
                },
                "pose":{
                    "pitch":-18.4849,
                    "roll":0.854,
                    "yaw":-4.2123
                },
                "frontal": true
            },
            "landmarks": {
                "faceContour": [[515,447],[517,491]...[872,436]],
                "noseBridge": [[710,419],[711,441]...[712,487]],
                "noseBall": [[680,519],[696,522]...[742,518]],
                "eyebrowRight": [[736,387],[768,376]...[854,394]],
                "eyebrowLeft": [[555,413],[578,391]...[679,391]],
                "eyeRight": [[753,428],[774,414]...[777,432]],
                "eyeRightCenter": [[786,423]],
                "eyeLeft": [[597,435],[617,423]...[619,442]],
                "eyeLeftCenter": [[630,432]],
                "mouthOuter": [[650,590],[674,572]...[675,600]],
                "mouthInner": [[661,587],[697,580]...[697,584]]
            }
        }
    ]
}

Response 400

HideShow

Invalid request.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 401

HideShow

API Token does not exist, is inactive, suspended, or deleted.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 403

HideShow

The API Token’s limit has been exceeded.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 405

HideShow

Invalid request method. Only POST is supported.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 406

HideShow

Invalid request path.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 413

HideShow

The image content is too large.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 415

HideShow

Invalid Content-Type

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 500

HideShow

Detection failed due to an unexpected condition.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 502

HideShow

The system is busy or unavailable.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

From Image File
POST`/v1/detections?type={type}&faceOption={faceOption}`

Example URI

POST https://dev.sighthoundapi.com/v1/detections?type=face,person&faceOption=gender,landmark,age,pose,emotion

URI Parameters

HideShow

type: string (optional) Example: face,person
Comma separated list of object categories to detect. Valid options are ‘all’, ‘face’, ‘person’. Default is ‘all’.
faceOption: string (optional) Example: gender,landmark,age,pose,emotion
For type ‘face’, additional detections can be performed by passing comma-separated list of values. Valid options are ‘gender’, ‘landmark’, ‘emotion’, ‘pose’, and ‘age’. Default returns face bounding box only.

Request

Request Body Attributes (JSON)

Binary Stream of an image file to upload

Headers

Content-Type: application/octet-stream
X-Access-Token: developersOwnCloudAccessToken

Body

< Image binary stream >

Response 200

Successful queries will return a 200 response with a JSON body describing all detected objects and the attributes of the processed image.

Response Attributes (JSON)

image (object): Information about the processed image.
- width (integer): Pixel width of source image
- height (integer): Pixel height of source image
- orientation (integer): EXIF code representing rotation and/or flipping of source Image needed to align the source photo with the detection coordinates. Only applicable for photos with EXIF data (e.g., photos taken on a mobile device). Defaults to 1 (no rotation or flip), else the value found in the image’s EXIF data and indicates that bounding vertices have been translated to match that coordinate space.
objects (array): An array of all detected objects. Each object contains:
- type (string): The label applied to the object, face or person.
- boundingBox (object): An object containing x, y, width, and height values defining the location of the object in the image. The top left pixel of the image represents coordinate (0,0).
- attributes (object): describes attributes of the detected Object
  - gender (string): The computed gender of the face. Always returns a value of either male or female. Returned only for objects with type=face.
  - genderConfidence (float): A normalized value between 0 - 1 that represents the confidence that the computed gender is correct.
  - age (integer): The estimated age of the face. Always returns a value from 0 to 100. Returned only for objects with type=face.
  - ageConfidence (float): A normalized value between 0 - 1 that represents the confidence of the estimated age.
  - emotion (string): The top computed emotion of the face. Returned only for objects with type=face.
  - emotionConfidence (float): A normalized value between 0 - 1 that represents the confidence of the top computed emotion. Returned only for objects with type=face.
  - emotionsAll (object): All computed emotions. Returned only for objects with type=face.
  - pose (object): The pitch, roll, and yaw of the detected pose. Returned only for objects with type=face.
  - frontal (string): Frontal is true if the face is looking at camera and false if it’s a profile view. Returned only for objects with type=face.
- landmarks (object): Contains x/y coordinates for several parts of the detected face. See the Sighthound Cloud Demo for placement.

Headers

Content-Type: application/json

Body

{
    "image": {
        "width": 1280, 
        "height": 960, 
        "orientation": 1
    },
    "objects": [
        {
            "type": "person",
            "boundingBox": { 
                "x": 363, 
                "y": 182, 
                "height": 778, 
                "width": 723
            } 
        },
        {
            "type": "face",
            "boundingBox": {
                "x": 508, 
                "y": 305, 
                "height": 406, 
                "width": 406
            },
            "attributes": {
                "gender": "male", 
                "genderConfidence": 0.9883, 
                "age":25,
                "ageConfidence": 0.2599,
                "emotion": "happiness",
                "emotionConfidence": 0.9943,
                "emotionsAll": {
                    "neutral": 0.0018,
                    "sadness": 0.0009,
                    "disgust": 0.0002,
                    "anger": 0.0003,
                    "surprise": 0,
                    "fear": 0.0022,
                    "happiness": 0.9943
                },
                "pose":{
                    "pitch":-18.4849,
                    "roll":0.854,
                    "yaw":-4.2123
                },
                "frontal": true
            },
            "landmarks": {
                "faceContour": [[515,447],[517,491]...[872,436]],
                "noseBridge": [[710,419],[711,441]...[712,487]],
                "noseBall": [[680,519],[696,522]...[742,518]],
                "eyebrowRight": [[736,387],[768,376]...[854,394]],
                "eyebrowLeft": [[555,413],[578,391]...[679,391]],
                "eyeRight": [[753,428],[774,414]...[777,432]],
                "eyeRightCenter": [[786,423]],
                "eyeLeft": [[597,435],[617,423]...[619,442]],
                "eyeLeftCenter": [[630,432]],
                "mouthOuter": [[650,590],[674,572]...[675,600]],
                "mouthInner": [[661,587],[697,580]...[697,584]]
            }
        }
    ]
}

Response 400

HideShow

Invalid request.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 401

HideShow

API Token does not exist, is inactive, suspended, or deleted.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 403

HideShow

The API Token’s limit has been exceeded.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 405

HideShow

Invalid request method. Only POST is supported.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 406

HideShow

Invalid request path.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 413

HideShow

The image content is too large.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 415

HideShow

Invalid Content-Type

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 500

HideShow

Detection failed due to an unexpected condition.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Response 502

HideShow

The system is busy or unavailable.

Response Attributes (JSON)

error (string): A human readable interpretation of the HTTP status code.
reason (string): A short description of what went wrong.
reasonCode (integer): A unique identifier for this error combining the HTTP status code and an internal error id.
details (string): An optional key which will contain additional information on the error if available.

Headers

Content-Type: application/json

Body

{
        "error": "ERROR MESSAGE"
        "reason": "reason for error",
        "reasonCode": 00000,
        "details": {
              "statusCode": 000,
              "statusMessage": "reason for error",
              "body": "description of error"
             }
    }

Introduction ¶

Usage Notes ¶

Detections ¶

Detect Objects ¶

Using Image URLPOST/v1/detections?type={type}&faceOption={faceOption}

Example URI

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

From Image FilePOST/v1/detections?type={type}&faceOption={faceOption}

Example URI

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Headers

Body

Introduction

Usage Notes

Using Image URL
POST`/v1/detections?type={type}&faceOption={faceOption}`

From Image File
POST`/v1/detections?type={type}&faceOption={faceOption}`