image

This module provides image processing and screenshot functionality for Android devices controlled via Total Control.
It supports capturing, comparing, and parsing screen images in BMP format, as well as memory-optimized operations and visual element detection.

Key Features:

  • Capture screenshots to file, memory, or device storage.
  • Find images on screen or in memory using exact or fuzzy matching.
  • Wait for target images to appear on screen (waitForImage).
  • Read and parse BMP headers, including custom TC Creator metadata (parseBMP).
  • Use memory slots (@0, @1, @2, @@) for faster repeated image operations.
  • Retrieve raw image data (pixels and metadata) from screen or memory.

Memory Screenshot Workflow:

  1. Save screenshot to memory with screenshotToMemory("@0")
  2. Perform operations like seekImage("@0", ...), getData(2, "@0"), etc.
  3. Clear memory with screenshotClearMemory("@0") or all with no args.

Image Search Capabilities:

  • seekImage: Supports multiple forms, including region-based, memory-based, and ImageInfo-based search with adjustable similarity.
  • seekImageByID: Use image mapping JSON to match pre-defined UI elements.
  • waitForImage: Wait for specific images to appear within timeout.

BMP Metadata Parsing:

Use parseBMP(filePath) to extract:

  • Standard BMP header (size, dimensions, color depth)
  • Optional TC Creator header (original screen region, package name, activity, resolution)

This module attaches Java and JavaScript methods to Device.prototype,
allowing direct method calls on device instances for image-related operations.

(inner) getData(type, locationopt) → {object|null}

Retrieve image data from the device based on the specified type.
Optionally, you can specify a memory location (e.g., "@0") to retrieve image data from a previously saved screenshot.

getData(tcConst.getData.Screen) : Get screenshot image data from the device
getData(tcConst.getData.Memory) : Get memory image data from the device

Supports:
getData(type)
getData(type,location)

Example
// Example 1: Get image data from memory location "@0"
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
sigmaDevice.screenshotToMemory("@0", 1, 1, 10, 10);
var ret = sigmaDevice.getData(2, "@0");
if (ret != null) {
    var info = ret.info;
    print("version: " + ret.version);
    print("datatype: " + ret.datatype);
    print("length: " + ret.length);
    print("width: " + info.width);
    print("height: " + info.height);
    print("data: " + ret.data);
} else {
    print("Failed to get data from Memory! " + lastError());
}


// Example 2: Get image data from last screenshot
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
var ret = sigmaDevice.getData(1);
if (ret != null) {
    var info = ret.info;
    print("version: " + ret.version);
    print("datatype: " + ret.datatype);
    print("length: " + ret.length);
    print("width: " + info.width);
    print("height: " + info.height);
    print("data: " + ret.data);
} else {
    print("Failed to get data from screenshot! " + lastError());
}

// Operation Result:
// If it executes successfully:
version: 1
datatype: 1
length: 300
width: 720
height: 1280
data: "[]"
Parameters:
number type

Two ways to get image data:

  • tcConst.getData.Screen，the constant value is 1
  • tcConst.getData.Memory，the constant value is 2
string location <optional>

The specified memory stores the location of the data; @0, @1 or @2 represents a fixed three memory locations, and "@@" represents the last memory screenshot.
Returns:
object | null

Returns an object containing image metadata and pixel data if successful; otherwise, returns null. Use lastError() to get the error message.
Right now, we only support grabbing memory and screen image data, after which we will consider expanding more other data, and the return value will be saved in the object.

  • version:int,the constant value is 1
  • datatype:int,the constant value is 1
  • length:array, the length of the array for image binary byte
  • info:json，including image width and height
  • data:array,image binary byte

(inner) parseBMP(imageName) → {object|null}

Parse a BMP image file and extract its header information.
If the image is created using the Total Control BMP Creator tool,
it will also include a tcHeader containing structured metadata.

"bmpHeader" contains the following information:

  • Magic: Fixed value 'BM', the type of bitmap file;
  • Size: Bitmap file size, for example: 102702;
  • startAddr: The starting position of the bitmap data, for example: 54;
  • Width: The width of the image;
  • Height: The height of the image;
  • bitsPerPixel: Bits per pixel color of the image, for example: 24;
  • imgSize: The size of the bitmap, in bytes, for example: 102084;
  • sizePerRow: The size of each line, for example: 564;

"tcHeader" contains the following information:

  • Magic: Fixed value 'TC;
  • Length: Length of the picture information added by the TC, in bytes, for example: 26;
  • Version: version, fixed value 1;
  • origWidth: The original width of the image, that is, the width of original image that used to intercept the image;
  • origHeight: The original height of the image, that is, the height of original image that used to intercept the image;
  • origX: X coordinate, Means that the image is taken from the X coordinate of the top left corner of the original image;
  • origY: Y coordinate, Means that the image is taken from the Y coordinate of the upper left corner of the original image;
  • jsonText: Additional information about the image, A for the activity name, the activity name of the image, P for the App package name, which APP the image comes from, and S for the resolution, for example: '{"A":"com.netease.cloudmusic.activity.LoginActivity","P":"com.netease.cloudmusic","S":70}'.
Example
// Example 1: Parse a BMP image created using Total Control BMP Creator
var { parseBMP } = require("image");
var ret = parseBMP('E:\\File\\test.bmp');
if (ret != null) {
    print("Congratulations, this API executes successfully. Return value: \n");
    printVar(ret);
} else {
    print("Sorry! " + lastError() + "\nReturn value: " + ret);
}

// Operation Result:
{
  bmpHeader: {
    magic: 'BM',
    size: 102702,
    startAddr: 54,
    width: 188,
    height: 182,
    bitsPerPixel: 24,
    imgSize: 102084,
    sizePerRow: 564
  },
  tcHeader: {
    magic: 'TC',
    length: 26,
    version: 1,
    origWidth: 720,
    origHeight: 1280,
    origX: 263,
    origY: 283,
    jsonText: '{"A":"com.netease.cloudmusic.activity.LoginActivity","P":"com.netease.cloudmusic","S":70}'
  }
}


// Example 2: Parse a standard BMP image without embedded Total Control metadata
var { parseBMP } = require("image");
var ret = parseBMP('E:\\File\\1539543927519.bmp');
if (ret != null) {
    print("Congratulations, this API executes successfully. Return value: \n");
    printVar(ret);
} else {
    print("Sorry! " + lastError() + "\nReturn value: " + ret);
}

// Operation Result:
 {
   bmpHeader: {
     magic: 'BM',
     size: 2764854,
     startAddr: 54,
     width: 720,
     height: 1280,
     bitsPerPixel: 24,
     imgSize: 2764800,
     sizePerRow: 2160
   },
   tcHeader: {
   }
 }
Parameters:
string imageName

The full path of the BMP image file to parse (e.g., "E:\File\test.bmp").
Returns:
object | null

If successful, returns a JSON object containing the keywords "bmpHeader" and "tcHeader".
"bmpHeader" means bitmap information header.
"tcHeader" indicates the additional header that Total Control adds to the bmp image after being processed by the Total Control BMP Creator tool.If the bmp image is not processed by the Total Control BMP Creator tool, the value of "tcHeader" is null.
If fail, returns null, and the lastError() function can get the specific error message.

(inner) screenshot(filePathOrLocation, filePathopt, imageType, topLeftXopt, topLeftYopt, bottomRightXopt, bottomRightYopt) → {number}

Capture a screenshot from the device and save it to the specified file path.
You can choose to capture the full screen or a specific region.
Optionally, use a memory location (e.g., "@0") if screenshotToMemory() was called previously. When we look for images, find colors, and recognize text, we can reuse the image data stored in the memory location, and it is faster to find, find, and recognize text.

Supports:
screenshot(filePath, imageType)
screenshot(filePath, imageType, topLeftX, topLeftY, bottomRightX, bottomRightY)
screenshot(location, filePath, imageType, topLeftX, topLeftY, bottomRightX, bottomRightY)

Note:
When we use the API "screenshot(location,filePath, imageType, topLeftX, topLeftY, bottomRightX, bottomRightY)":
First we need to make sure that the memory location @0, @1 or @2 is available;
To ensure that the memory location is available, you can first use the API "screenshotToMemory" to specify the memory location where you want to store the data. The life cycle of in-memory data is the same as the life cycle of the Total Control application on the mobile phone. That is to say, as long as we do not close the Total Control application on the mobile terminal, we only need to use the “screenshotToMemory” API for each memory location to specify the memory location.
For example, take a screenshot to memory and specify the memory location "@0" to store the data.
var { Device } = require("sigma/device");
var sigmaDevice = Device.getMain();
sigmaDevice.screenshotToMemory("@0");

Example
// Example 1: Capture a screenshot of a specific region and save as JPG
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret = sigmaDevice.screenshot("D:/img/image.jpg", sigmaConst.IMG_JPG, 20, 20, 300, 300);
    if (ret === 0) {
        print("Congratulations, this API executes successfully.\nReturn value: " + ret);
    } else {
        print("Sorry! " + lastError() + "\nReturn value: " + ret);
    }
} else {
    print("Failed to get the master device object");
}


// Example 2: Use screenshotToMemory and screenshot with memory location
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
sigmaDevice.screenshotToMemory("@0");

var ret = sigmaDevice.screenshot("@0", "D:/img/image.jpg", sigmaConst.IMG_JPG, 20, 20, 300, 300);
if (ret === 0) {
    print("Congratulations, this API executes successfully.\nReturn value: " + ret);
} else {
    print("Sorry! " + lastError() + "\nReturn value: " + ret);
}

// Operation Result (if successful):
Congratulations, this API executes successfully.
Return value: 0
Parameters:
string filePathOrLocation

The path to save the image (e.g., "D:/img/image.jpg")
or the specified memory stores the location of the data; @0, @1 or @2 represents a fixed three memory locations, and "@@" represents the last memory screenshot.
string filePath <optional>

The path to save the image. Required only when the first argument is a memory location.
number imageType

Image format (e.g., sigmaConst.IMG_JPG, sigmaConst.IMG_PNG, sigmaConst.IMG_BMP).
number topLeftX <optional>

Optional. The x-coordinate of the top-left corner of the capture area, do not fill in the coordinate value, indicating a full screen screenshot.
number topLeftY <optional>

Optional. The y-coordinate of the top-left corner of the capture area, do not fill in the coordinate value, indicating a full screen screenshot.
number bottomRightX <optional>

Optional. The x-coordinate of the bottom-right corner of the capture area, do not fill in the coordinate value, indicating a full screen screenshot.
number bottomRightY <optional>

Optional. The y-coordinate of the bottom-right corner of the capture area, do not fill in the coordinate value, indicating a full screen screenshot.
Returns:
number

This function returns 0 on success or -1 on failure. Specific error information can be obtained by the lastError() function.

(inner) screenshotClearMemory(locationopt) → {number}

Clear data stored in memory, clear the data that the API ”screenshotToMemory” saves into memory. It can either clear the data of the specified memory location or clear the data of all storage locations in the memory.
That is, clear the data saved by the "screenshotToMemory" API to the memory location.

Supports:
screenshotClearMemory(location)
screenshotClearMemory()

Example
// Example 1: Clear a specific memory location (e.g., "@0")
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret = sigmaDevice.screenshotClearMemory("@0");
    if (ret === 0) {
        print("Congratulations, this API executes successfully.\nReturn value: " + ret);
    } else {
        print("Sorry! " + lastError() + "\nReturn value: " + ret);
    }
} else {
    print("Failed to get the master device object");
}


// Example 2: Clear all memory locations
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
var ret = sigmaDevice.screenshotClearMemory();
if (ret === 0) {
    print("All screenshot memory locations cleared.");
} else {
    print("Failed to clear memory: " + lastError());
}

// Operation Result:
// If it executes successfully, it will print:
Congratulations, this API executes successfully.
Return value: 0
Parameters:
string location <optional>

The specified memory stores the location of the data; @0, @1 or @2 represents a fixed three memory locations, and "@@" represents the last memory screenshot.
Returns:
number

Returns 0 if successful; otherwise, returns a non-zero error code. Use lastError() to retrieve the error message.

(inner) screenshotToDevice(filePath, imageType, topLeftXopt, topLeftYopt, bottomRightXopt, bottomRightYopt) → {number}

Capture a screenshot on the device and save it directly to a file on the device itself.
You can capture the full screen or specify a rectangular region.

Supports:
screenshotToDevice(filePath, imageType, topLeftX, topLeftY, bottomRightX, bottomRightY)
screenshotToDevice(filePath, imageType)

Example
// Example 1: Capture full screen and save to device as PNG
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret = sigmaDevice.screenshotToDevice("/sdcard/aa/image.png", tcConst.IMG_PNG);
    if (ret === 0) {
        print("Congratulations, this API executes successfully.\nReturn value: " + ret);
    } else {
        print("Sorry! " + lastError() + "\nReturn value: " + ret);
    }
} else {
    print("Failed to get the master device object");
}


// Example 2: Capture a region and save to device
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
var ret = sigmaDevice.screenshotToDevice("/sdcard/aa/image1.jpg", tcConst.IMG_JPG, 100, 100, 500, 500);
if (ret === 0) {
    print("Region screenshot saved to device successfully.");
} else {
    print("Screenshot failed: " + lastError());
}

// Operation Result:
// If it executes successfully:
Congratulations, this API executes successfully.
Return value: 0
Parameters:
string filePath

Save the full path of the screenshot image, such as "/data/local/tmp/tmp.bmp".
number imageType

Image format (e.g., tcConst.IMG_PNG, tcConst.IMG_JPG, tcConst.IMG_BMP).
number topLeftX <optional>

Optional. X-coordinate of the top-left corner of the region to capture, do not fill in the coordinate value, indicating a full screen screenshot.
number topLeftY <optional>

Optional. Y-coordinate of the top-left corner of the region to capture, do not fill in the coordinate value, indicating a full screen screenshot.
number bottomRightX <optional>

Optional. X-coordinate of the bottom-right corner of the region to capture, do not fill in the coordinate value, indicating a full screen screenshot.
number bottomRightY <optional>

Optional. Y-coordinate of the bottom-right corner of the region to capture, do not fill in the coordinate value, indicating a full screen screenshot.
Returns:
number

This function returns 0 on success or -1 on failure. Specific error information can be obtained by the lastError() function.

(inner) screenshotToMemory(location, topLeftXopt, topLeftYopt, bottomRightXopt, bottomRightYopt) → {number}

Take a screenshot and save the image data to the specified memory location. Total Control only provides 3 memory locations @0, @1 or @2, and each memory location can only store at most one image data. The image data can be stored in the same memory location multiple times,however, the memory location @0, @1 or @2 only saves the last stored image data.
The life cycle of the in-memory data is the same as the life cycle of the Total Control application on the mobile phone. That is to say, when we close the Total Control application on the mobile terminal, the image data stored in the memory is gone.
When we look for images, find colors, and recognize text, we can reuse the image data stored in the memory location, and it is faster to find, find, and recognize text.

Supports:
screenshotToMemory(location)
screenshotToMemory(location, topLeftX, topLeftY, bottomRightX,bottomRightY)

Example
// Example 1: Capture full screen to memory location "@0"
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
var ret = sigmaDevice.screenshotToMemory("@0");
if (ret === 0) {
    print("Successfully screenshot and saved to memory");
} else {
    print(lastError());
}


// Example 2: Capture a specific region to memory location "@1"
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
var ret = sigmaDevice.screenshotToMemory("@1", 100, 100, 400, 400);
if (ret === 0) {
    print("Successfully captured region to memory");
} else {
    print("Error: " + lastError());
}

// Operation Result:
// If it executes successfully:
Successfully screenshot and saved to memory
Parameters:
string location

The specified memory stores the location of the data; @0, @1 or @2 represents a fixed three memory locations, and "@@" represents the last memory screenshot.
number topLeftX <optional>

Optional. The x-coordinate of the top-left corner of the region to capture, do not fill in the coordinate value, indicating a full screen screenshot.
number topLeftY <optional>

Optional. The y-coordinate of the top-left corner of the region to capture, do not fill in the coordinate value, indicating a full screen screenshot.
number bottomRightX <optional>

Optional. The x-coordinate of the bottom-right corner of the region to capture, do not fill in the coordinate value, indicating a full screen screenshot.
number bottomRightY <optional>

Optional. The y-coordinate of the bottom-right corner of the region to capture, do not fill in the coordinate value, indicating a full screen screenshot.
Returns:
number

This function returns 0 on success or -1 on failure. Specific error information can be obtained by the lastError() function.

(inner) seekImage(…args) → {Object|false|null}

Find the specified image in full screen or specified area, support searching for the specified image on the screen and searching for the specified image in memory, and support similar search. If the target image is found, it returns the center coordinates and coordinate area of the found image.
when we use memory to find a image, first you need to store the image to a specified location in the memory. The available memory is @ 0, @ 1 or @ 2. When we look for images, find colors, and recognize text, we can reuse the image data stored in the memory location. This will be faster. For how to save image to memory, please refer to screenshotToMemory API. Image saved to memory do not always exist, and their life cycle is the life cycle of the Total Control application. This means that as long as we do not close the Total Control application on the mobile phone, the image data saved to the memory will always be available.
When we need to use this API to manipulate the image data stored in the specified memory location, we use the API with the "location" parameter

Note:
Although we support taking screenshots on one phone and searching for image on another phone, the height and width ratios of the two phones must be the same. In addition, if the screenshot is taken in landscape orientation, you must also ensure that the phone is in landscape orientation when looking for pictures; similarly, portrait screenshots must also be found in portrait orientation.
The current version only supports bmp. In future development plans, we will gradually support various image formats.

Example
// Example 1: Find the specified image in full screen
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret = sigmaDevice.seekImage("D:\\test.bmp");
    if (ret == null) {
        print("Sorry, execute failed! The error is: " +  lastError());
    } else if (ret == false) {
        print("Sorry, did not find the specified image!");
    } else {
        if (ret.rect) {
            var arr = [ret.rect[0],ret.rect[1],ret.rect[2],ret.rect[3]];
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y + ", rect: " + JSON.stringify(arr));
        } else {
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y );
        }
    }
}


// Example 2: Find the specified image in the specified area
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret = sigmaDevice.seekImage(13, 422, 583, 986, "D:\\test.bmp");
    if (ret == null) {
        print("Sorry, execute failed! The error is: " +  lastError());
    } else if (ret == false) {
        print("Sorry, did not find the specified image!");
    } else {
        if (ret.rect) {
            var arr=[ret.rect[0],ret.rect[1],ret.rect[2],ret.rect[3]];
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y + ", rect: " + JSON.stringify(arr));
        } else {
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y );
        }
    }
}


// Example 3: Region search + similarity 0.6
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret = sigmaDevice.seekImage(13, 422, 583, 986, "D:\\test.bmp", 0.6);
    if (ret == null) {
        print("Sorry, execute failed! The error is: " +  lastError());
    } else if (ret == false) {
        print("Sorry, did not find the specified image!");
    } else {
        if (ret.rect) {
            var arr = [ret.rect[0],ret.rect[1],ret.rect[2],ret.rect[3]];
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y + ", rect: " + JSON.stringify(arr));
        } else {
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y );
        }
    }
}


// Example 4: Full screen + similarity 0.6 + grayscale
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret = sigmaDevice.seekImage("D:\\test.bmp", 0.6, true);
    if (ret == null) {
        print("Sorry, execute failed! The error is: " +  lastError());
    } else if (ret == false) {
        print("Sorry, did not find the specified image!");
    } else {
        if (ret.rect) {
            var arr = [ret.rect[0],ret.rect[1],ret.rect[2],ret.rect[3]];
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y + ", rect: " + JSON.stringify(arr));
        } else {
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y );
        }
    }
}


// Example 5: Match from memory screenshot
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret1 = sigmaDevice.screenshotToMemory("@0");
    if (ret1 == 0) {
        var ret = sigmaDevice.seekImage("@0", "D:\\test.bmp");
        if (ret == null) {
            print("Sorry, execute failed! The error is: " +  lastError());
        } else if (ret == false) {
            print("Sorry, did not find the specified image!");
        } else {
            if (ret.rect) {
                var arr = [ret.rect[0],ret.rect[1],ret.rect[2],ret.rect[3]];
                print("Image found, Center x: " + ret.x + ", Center y: " + ret.y + ", rect: " + JSON.stringify(arr));
            } else {
                print("Image found, Center x: " + ret.x + ", Center y: " + ret.y );
            }
        }
    }
}


// Example 6: Memory + region + sim = 0.7
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret1 = sigmaDevice.screenshotToMemory("@0");
    if (ret1 == 0) {
        var ret = sigmaDevice.seekImage("@0", 13, 422, 583, 986, "D:\\test.bmp", 0.7);
        if (ret == null) {
            print("Sorry, execute failed! The error is: " +  lastError());
        } else if (ret == false) {
            print("Sorry, did not find the specified image!");
        } else {
            if (ret.rect) {
                var arr = [ret.rect[0],ret.rect[1],ret.rect[2],ret.rect[3]];
                print("Image found, Center x: " + ret.x + ", Center y: " + ret.y + ", rect: " + JSON.stringify(arr));
            } else {
                print("Image found, Center x: " + ret.x + ", Center y: " + ret.y );
            }
        }
    }
}


// Example 7: Use ImageInfo object
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var imageInfo = newImageInfo();
    imageInfo.imagePath="D:\\test.bmp";
    imageInfo.x=3;
    imageInfo.y=3;
    imageInfo.width=sigmaDevice.width - 10;
    imageInfo.height=sigmaDevice.height - 10;
    imageInfo.srcResolutionX=720;
    imageInfo.srcResolutionY=1280;
    var ret = sigmaDevice.seekImage(imageInfo);
    if (ret == null) {
        print("Sorry, execute failed! The error is: " +  lastError());
    } else if (ret == false) {
        print("Sorry, did not find the specified image!");
    } else {
        if (ret.rect) {
            var arr = [ret.rect[0],ret.rect[1],ret.rect[2],ret.rect[3]];
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y + ", rect: " + JSON.stringify(arr));
        } else {
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y );
        }
    }
}


// Example 8: ImageInfo + sim = 0.7
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var imageInfo = newImageInfo();
    imageInfo.imagePath="D:\\test.bmp";
    imageInfo.x=5;
    imageInfo.y=5;
    imageInfo.width=sigmaDevice.width - 5;
    imageInfo.height=sigmaDevice.height - 10;
    imageInfo.srcResolutionX=720;
    imageInfo.srcResolutionY=1280;
    var ret = sigmaDevice.seekImage(imageInfo, 0.7);
    if (ret == null) {
        print("Sorry, execute failed! The error is: " +  lastError());
    } else if (ret == false) {
        print("Sorry, did not find the specified image!");
    } else {
        if (ret.rect) {
            var arr = [ret.rect[0],ret.rect[1],ret.rect[2],ret.rect[3]];
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y + ", rect: " + JSON.stringify(arr));
        } else {
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y );
        }
    }
}


// Example 9: Use options object for similarity
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret = sigmaDevice.seekImage("D:\\test.bmp", 10, 10, 700, 1024, {sim: 0.5});
    if (ret == null) {
        print("Sorry, execute failed! The error is: " +  lastError());
    } else if (ret == false) {
        print("Sorry, did not find the specified image!");
    } else {
        if (ret.rect) {
            var arr = [ret.rect[0],ret.rect[1],ret.rect[2],ret.rect[3]];
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y + ", rect: " + JSON.stringify(arr));
        } else {
            print("Image found, Center x: " + ret.x + ", Center y: " + ret.y );
        }
    }
}
Parameters:
any args <repeatable>

Flexible arguments depending on usage.

Supported Forms:

  1. seekImage(topLeftX, topLeftY, bottomRightX, bottomRightY, imageName, sim, beGray)

    • Search for imageName within a screen region.
    • Optionally define sim (similarity, e.g. 0.6~1.0) and beGray (boolean to enable grayscale).

  2. seekImage(location, topLeftX, topLeftY, bottomRightX, bottomRightY, imageName, sim, beGray)

    • Search in a previously captured memory image (via screenshotToMemory("@0")), within a region.
    • Useful for matching against stored screen captures.

  3. seekImage(imageInfo, sim)

    • Match using a predefined ImageInfo object (created via newImageInfo()).
    • You can set resolution, region, and imagePath in the object, and optionally set similarity.

  4. seekImage(imageName, x1, y1, x2, y2, options)

    • Match in region using options object: { sim: 0.5, beGray: true }
Returns:
Object | false | null
  • Object if matched: { x: number, y: number, rect?: [x1, y1, x2, y2] }
  • false if image not found
  • null if execution failed (check lastError())

(inner) seekImageByID(imageID, simopt) → {Object|null}

Search for the specified image on the device screen using an image ID defined in the image_mapping.json configuration file.
Optionally, you can specify the similarity threshold for matching.

Supports:
seekImageByID(imageID)
seekImageByID(imageID, sim)

Set the configuration file:
Please prepare the file {Documents}\resources\images\image_mapping.json in "My Documents". The role of the image_mapping.json file is to pre-define parameters in the configuration file.
The fixed format of the file is:
{
'IDMapImageFile':
[
{'ID':'juhuasuan', APP:'weixin', REGION: {x:0, y:0, width:719, height: 1279, srcResolutionX: 720, srcResolutionY: 1280}, IMAGEFILENAME:'jianghu.bmp'},
{'ID':'AAA', APP:'weixin', REGION: {x:0, y:0, width:1440, height: 2560, srcResolutionX: 1440, srcResolutionY: 2560}, IMAGEFILENAME:'1440small.png'}
]
}

The meaning of the parameters in the file:

  • ID: ID of the image, user-defined;
  • APP: Which application the icon comes from, in fact, is a subpath, that is, the subfolder where the icon is located. If the value is weixin, the path of the subfolder is "{Documents}\resources\images\weixin";
  • REGION: Json object, including the following fields:
    -- x: X coordinate of the upper left corner of the screen search range
    -- y: Y coordinate of the upper left corner of the screen search range
    -- width: The width of the screen search range
    -- srcResolutionX: The image to be searched is obtained from which resolution of the phone, this parameter is the width of the resolution
    -- srcResolutionY: The image to be searched for is obtained from the resolution of the phone, this parameter is the height of the resolution
  • IMAGEFILENAME: the name of the image;
Example
// Example 1: Search for an image using only the image ID
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    var ret = sigmaDevice.seekImageByID("AAA0");
    if (ret != null) {
        print("Congratulations, this API executes successfully.\nReturn value: ");
        print("Center X: " + ret.x);
        print("Center Y: " + ret.y);
    } else {
        print("Sorry! " + lastError() + "\nReturn value: " + ret);
    }
} else {
    print("Failed to get the master device object");
}


// Example 2: Search with custom similarity threshold (e.g., 0.85)
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
var ret = sigmaDevice.seekImageByID("AAA0", 0.85);
if (ret != null) {
    print("Congratulations, this API executes successfully.\nReturn value: ");
    print("Center X: " + ret.x);
    print("Center Y: " + ret.y);
} else {
    print("Image not found: " + lastError());
}

// Operation Result:
// If successful, returns:
Congratulations, this API executes successfully.
Return value:
Center X: 446
Center Y: 336
Parameters:
string imageID

The image identifier defined in the image mapping file (e.g., "AAA0").
number sim <optional>

Similarity, the default is 1.0, the similarity range is [0.0, 1.0]. 1 means the image is completely matched. The smaller the value of sim, the lower the requirement for similarity.
Returns:
Object | null

If successful, returns the center point coordinate object of the found image, which can be used to obtain the coordinates of the center point, such as coord.x; coord.y; otherwise, it returns null. Specific error information can be obtained by the lastError() function.

(inner) waitForImage(location, path, sim, timeout) → {Object|null}

Wait for the target image to appear on the screen.
when we use memory to find a image, first you need to store the image to a specified location in the memory. The available memory is @ 0, @ 1 or @ 2. When we look for images, find colors, and recognize text, we can reuse the image data stored in the memory location. This will be faster. For how to save image to memory, please refer to screenshotToMemory API. Image saved to memory do not always exist, and their life cycle is the life cycle of the Total Control application. This means that as long as we do not close the Total Control application on the mobile phone, the image data saved to the memory will always be available.

Supports:
waitForImage(location, path, sim, timeout)
waitForImage(path, sim, timeout)

Example
var { Device } = require("sigma/device");
var image = require("sigma/image");
// Get the current master device object
var sigmaDevice = Device.getMain();
if (sigmaDevice != null) {
    // Execute waitForImage with specified image path, similarity 0.8, and timeout 5000ms
    var ret = sigmaDevice.waitForImage("E:\\File\\img\\Login.bmp", 0.8, 5000);

    if (ret != null) {
        print("Congratulations, this API executes successfully.\nReturn value:");
        print("Center X: " + ret.x);
        print("Center Y: " + ret.y);
    } else {
        print("Sorry! " + lastError() + "\nReturn value: " + ret);
    }
} else {
    print("Failed to get the master device object");
}

//If it executes successfully, it will return:
Congratulations, this API executes successfully.
Return value:
Center X: 446
Center Y: 336
Parameters:
string location

The specified memory stores the location of the data; @0, @1 or @2 represents a fixed three memory locations, and "@@" represents the last memory screenshot.
string path

The absolute path of the image, the image must be in bmp format, For example: "d:/image.bmp".
number sim

Similarity, the default is 1.0, the similarity range is [0.0, 1.0]. 1 means the image is completely matched. The smaller the value of sim, the lower the requirement for similarity.
number timeout

Timeout (milliseconds).
Returns:
Object | null

If successful, returns the center point coordinate object of the found image, which can be used to obtain the coordinates of the center point, such as coord.x; coord.y; otherwise, it returns null. Specific error information can be obtained by the lastError() function.