Lens Scripting API
    Preparing search index...

    Class TensorMath

    Namespace for mathematical operations on tensors. Useful with MLComponent.

    Tensor broadcasting rules*

    Tensor (channels, width, height) op Tensor (1, 1, 1) = the same as applying op with scalar

    Tensor (channels, width, height) op Tensor (channels, 1, 1) = the same as applying op per channel

    // Convert a texture to an array of grayscale values
    // @input Asset.Texture texture

    // Desired width and height of the grayscale texture
    // You can instead use Texture.getWidth() and Texture.getHeight() for original size
    var width = 64;
    var height = 64;

    var data = new Uint8Array(height * width);
    var shape = new vec3(width, height, 1);

    // Get the texture pixels as grayscale values (0-255)
    TensorMath.textureToGrayscale(script.texture, data, shape);
    Index

    Constructors

    Methods

    • Adds a scalar value to each element of inTensor and puts the result into outTensor.

      Parameters

      • inTensor: Float32Array
      • scalar: number
      • outTensor: Float32Array

      Returns void

    • Adds inTensorA to inTensorB and puts the result into outTensor. See the broadcasting rules for elementwise operations.

      Parameters

      • inTensorA: Float32Array
      • inShapeA: vec3
      • inTensorB: Float32Array
      • inShapeB: vec3
      • outTensor: Float32Array

      Returns void

    • Converts inTensor from the amplitude scale to the decibel scale.

      Parameters

      • inTensor: Float32Array
      • outTensor: Float32Array

      Returns void

    • Blurs an image tensor using the box filter.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • kernelSize: vec2
      • anchor: vec2
      • normalize: boolean
      • borderType: BorderType
      • outTensor: Float32Array

      Returns void

    • Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • scores: Float32Array
      • scoreThreshold: number
      • iouThreshold: number
      • outTensor: Uint32Array

      Returns number

    • Applies a fixed-level threshold to each array element.

      Parameters

      • inTensor: Float32Array
      • threshold: number
      • maxValue: number
      • type: ThresholdMethod
      • outTensor: Float32Array

      Returns void

    • Approximates a polygonal curve with the specified precision.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • epsilon: number
      • closed: boolean
      • outTensor: Float32Array

      Returns number

    • Returns the indices of the maximum values along an the channels of inTensor, with the specified inShape. The result is put into outTensor.

      If inShape = {width, height, channels}, then the shape of outTensor should be {1, 2, channels}.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • outTensor: Uint32Array

      Returns void

    • Similar to numpy.argsort(), but in TensorMath, we have no kind and order parameters. Also, tensor is always 3D, and axis can be equal to 0(x), 1(y) or 2(z).

      Parameters

      • inTensor: Float32Array
      • shape: vec3
      • axis: number
      • outTensor: Uint32Array

      Returns void

    • Applies a list of indices of a tensor in sorted order of their corresponding values in the tensor to the given outTensor. Only indices whose corresponding values are not equal to 0 in the provided mask will be returned. Indices from the original tensor, before applying the mask, will be returned in the provided array. Returns the size of the list of indices applied to the outTensor (note: this will be equal to the number of non-zero values provided in the mask). eg:

      var inTensor = [5.0, 2.0, 1.0, 3.0, 7.0, -1.0, -5.0]; 
      var mask = [1.0, 1.0, 1.0, 1.0, 1.0, 0.0, 0.0];
      var outTensor = new Uint32Array(7);
      var order = SortOrder.Ascending;
      var size = TensorMath.argSortMasked(inTensor, mask, outTensor, order);
      print(outTensor.subarray(size)); // Expected: [2, 1, 3, 0, 4]

      Parameters

      • inTensor: Float32Array
      • mask: Float32Array
      • outTensor: Uint32Array
      • order: SortOrder

      Returns number

    • Clamps all values of inTensor between minVal and maxVal, and puts the result into outTensor.

      Parameters

      • inTensor: Float32Array
      • minVal: number
      • maxVal: number
      • outTensor: Float32Array

      Returns void

    • Concatenates inTensorA and inTensorB along the specified axis and puts the result to the outTensor. Input arrays have to have same dimension along other 2 axes.

      Parameters

      • inTensorA: Float32Array
      • inShapeA: vec3
      • inTensorB: Float32Array
      • inShapeB: vec3
      • axis: number
      • outTensor: Float32Array

      Returns void

    • Dilates an image by using a specific structuring element.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • kernelTensor: Float32Array
      • kernelShape: vec3
      • anchor: vec2
      • iterations: number
      • borderType: BorderType
      • borderValue: vec4
      • outTensor: Float32Array

      Returns void

    • Divides inTensorA by inTensorB and puts the result into outTensor. See broadcasting rules for elementwise operations.

      Parameters

      • inTensorA: Float32Array
      • inShapeA: vec3
      • inTensorB: Float32Array
      • inShapeB: vec3
      • outTensor: Float32Array

      Returns void

    • Draws a line segment connecting two points.

      Parameters

      • imgTensor: Float32Array
      • imgShape: vec3
      • point1: vec2
      • point2: vec2
      • color: vec4
      • thickness: number
      • lineType: number
      • shift: number

      Returns void

    • Erodes an image by using a specific structuring element.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • kernelTensor: Float32Array
      • kernelShape: vec3
      • anchor: vec2
      • iterations: number
      • borderType: BorderType
      • borderValue: vec4
      • outTensor: Float32Array

      Returns void

    • Fills a convex polygon.

      Parameters

      • imgTensor: Float32Array
      • imgShape: vec3
      • pointsTensor: any
      • pointsShape: vec3
      • color: vec4
      • lineType: number
      • shift: number

      Returns void

    • Fills a polygon. Note that you should pass an Array of Int32Array-s (polygonsTensors).

      Parameters

      • imgTensor: Float32Array
      • imgShape: vec3
      • polygonsTensors: any[]
      • color: vec4
      • lineType: number
      • shift: number
      • offset: vec2

      Returns void

    • Returns contours. Note that: contours are sorted from the largest to the smallest; 2) We cannot return Array of TypedArray-s, so all contours data is stored in single outTensor and Array with sizes of each contour is returned.

      Parameters

      • inTensor: Uint8Array
      • inShape: vec3
      • mode: number
      • method: number
      • offset: vec2
      • outTensor: any

      Returns number[]

    • Finds minimum distances between each 2D point from one array, to 2D points in another array.

      from - Float32Array of size ("from" point count * 2). 2D points from which min distances should be found

      fromShape - Should be {2, "from" point count, 1}

      to - Float32Array of size ("to" point count * 2). 2D points to which min distances should be found

      toShape - Should be {2, "to" point count, 1}

      output - Float32Array of size ("from" point count). For each point in the from array, the minimum distance to points from the to array will be written to this array.

      Parameters

      • from: Float32Array
      • fromShape: vec3
      • to: Float32Array
      • toShape: vec3
      • output: Float32Array

      Returns void

    • Looks for contour points in a grayscale texture, or any one-channel tensor.

      grayscaledTexture - Float32Array of size (width * height). Grayscale texture or one-channel tensor to evaluate

      textureShape - Should be {width, height, 1}

      threshold - Quality value threshold for found contour points

      Each contour point found must satisfy these conditions:

      1. The point's quality value should be >= threshold

      2. The number of points which have values < threshold and lie in the rectangle with left corner vec2(x-winSize, y-winSize) and right corner vec2(x+winSize-1, y+winSize-1) should be <= maxNearCount

      outTensor - Float32Array where results are written. Found contour points are written in the format: x0, y0, x1, y1, etc. The number of points found will not exceed outTensor's size / 2.

      Parameters

      • grayscaledTexture: Float32Array
      • textureShape: vec3
      • threshold: number
      • winSize: number
      • maxNearCount: number
      • outTensor: Float32Array

      Returns number

    • Parameters

      Returns void

    • Calculates the length of all vectors in an array. Vectors can be of any dimension count.

      vectors - Float32Array of size (vector dimension count * vector count). Vectors to measure the length of

      vectorsShape - Should be {vector dimension count, vector count, 1}

      output - Float32Array of size (vector count). For each vector in vectors, its length will be written to this array

      Parameters

      • vectors: Float32Array
      • vectorsShape: vec3
      • output: Float32Array

      Returns void

    • Checks for each 2D point whether it is inside of a rectangle.

      points - Float32Array of size (point count * 2). 2D points to check

      pointsShape - Should be {2, point count, 1}

      rect - 2D rectangle points will be checked against

      output - Uint8Array of size (point count). For each point, this will be filled with 1 if the point is inside the rectangle, or 0 otherwise.

      Parameters

      • points: Float32Array
      • pointsShape: vec3
      • rect: Rect
      • output: Uint8Array

      Returns void

    • Places the maximum values of inTensor into outTensor.

      outTensor should have the shape {1, 1, channels}.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • outTensor: Float32Array

      Returns void

    • From the input points, finds the two points with the maximum distance between them and returns this distance. Works with points of any dimension count.

      points - Float32Array of size (point dimension count * point count). Points to find the max distance between.

      pointsShape - Should be {point dimension count, point count, 1}

      Parameters

      • points: Float32Array
      • pointsShape: vec3

      Returns number

    • If we consider the tensor as a 3D array, this function finds the indexes of the maximum element in each subarray of size window.

      tensor - Float32Array of size (width * height * depth). 3D array of input data

      tensorShape - Should be {width, height, depth}

      window - Size of each subarray, in each of which will be found the index of the max element

      output - Float32Array of size (width * height * depth). The index of the max value will be written into this array for each subarray.

      Parameters

      • tensor: Float32Array
      • tensorShape: vec3
      • window: vec3
      • output: Uint32Array

      Returns void

    • Places the minimum values of inTensor into outTensor.

      outTensor should have the shape {1, 1, channels}.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • outTensor: Float32Array

      Returns void

    • Return TensorMath's format of RotatedRect.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3

      Returns RotatedRect

    • If we consider the tensor as a 3D array, this function finds the indexes of the minimum element in each subarray of size window.

      tensor - Float32Array of size (width * height * depth). 3D array of input data

      tensorShape - Should be {width, height, depth}

      window - Size of each subarray, in each of which will be found the index of the min element

      output - Float32Array of size (width * height * depth). The index of the min value will be written into this array for each subarray.

      Parameters

      • tensor: Float32Array
      • tensorShape: vec3
      • window: vec3
      • output: Uint32Array

      Returns void

    • Multiply a matrix by a set of points through an efficient batch operation.

      Parameters

      • pointsTensor: Float32Array
      • pointsShape: vec3
      • matrix: mat4
      • outTensor: Float32Array

      Returns void

    • Multiplies each element of inTensor by a scalar value and puts the result into outTensor.

      Parameters

      • inTensor: Float32Array
      • scalar: number
      • outTensor: Float32Array

      Returns void

    • Multiplies inTensorA and inTensorB and puts the result into outTensor. See broadcasting rules for elementwise operations.

      Parameters

      • inTensorA: Float32Array
      • inShapeA: vec3
      • inTensorB: Float32Array
      • inShapeB: vec3
      • outTensor: Float32Array

      Returns void

    • Stabilizes image objects between two consecutive frames caused by camera or object movement. Results are written to the points array.

      prevGrayscale - Uint8Array of size (width * height). Grayscale data of previous frame

      grayscale - Uint8Array of size (width * height). Grayscale data of current frame

      textureShape - Should be {width, height, 1}

      prevPoints - Float32Array of size (point count * 2). Positions of 2D points on the previous frame

      points - Float32Array of size (point count * 2). Results are written here - positions of 2D points on the current frame

      pointsShape - Should be {2, point count, 1}

      winSize - Size of the search window for each pyramid level

      maxLevel - Maximal pyramid level number, with zero based index

      maxCount - Terminate if iteration count exceeds maxCount

      epsilon - Terminate if window movement is less than epsilon

      Parameters

      • prevGrayscale: Uint8Array
      • grayscale: Uint8Array
      • textureShape: vec3
      • prevPoints: Float32Array
      • points: Float32Array
      • pointsShape: vec3
      • winSize: vec2
      • maxLevel: number
      • maxCount: number
      • epsilon: number

      Returns void

    • Rearranges the inTensor of inShape according to the desired ordering and puts the result into outTensor.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • permuteAxis: vec3
      • outTensor: Float32Array

      Returns void

    • Sorts 2D points by polar angle relative to the center point.

      inTensor - Float32Array with size (point count * 2). 2D points to sort

      tensorShape - Should be {2, point count, 1}

      center - Center point to use for polar angle sorting

      Parameters

      • inTensor: Float32Array
      • tensorShape: vec3
      • center: vec2

      Returns void

    • Raises elements of inTensor to the power of val, and puts the results into outTensor.

      Parameters

      • inTensor: Float32Array
      • val: number
      • outTensor: Float32Array

      Returns void

    • Converts inTensor from the power scale to the decibel scale.

      Parameters

      • inTensor: Float32Array
      • topDb: number
      • outTensor: Float32Array

      Returns void

    • Project 3D points into 2D space using an efficient batch operation.

      Parameters

      • pointsTensor: Float32Array
      • pointsShape: vec3
      • projectionMatrix: mat4
      • outTensor: Float32Array

      Returns void

    • Duplicates inTensor elements and store result in the outTensor.

      Axis specifies the number of repeats along the axis e.g:

      axis(1, 1, 1): outTensor will be the same as inTensor

      axis(2, 1, 2): inTesnor = [1 2 3 | 4 5 6], inShape = [3 2 1] =>

      outTensor = [[1 1 2 2 3 3 | 4 4 5 5 6 6] [1 1 2 2 3 3 | 4 4 5 5 6 6]] outShape = [6 2 2]

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • axis: vec3
      • outTensor: Float32Array

      Returns void

    • Applies a rotation to each point in a set of 3D points, and places the results in outPoints.

      points - Float32Array of size (point count * 3). Points to rotate

      pointsShape - Should be {3, point count, 1}

      rotation - Quaternion rotation to apply

      outPoints - Float32Array of size (point count * 3). Resulting rotated points are placed here

      Parameters

      • points: Float32Array
      • pointsShape: vec3
      • rotation: quat
      • outPoints: Float32Array

      Returns void

    • Smooths a polygon formed by input points. Works with points of any dimension count, for example 2 for 2D points or 3 for 3D points. Results are written to outTensor.

      inTensor - Float32Array of size (point dimension count * point count). Points of polygon to smooth out

      tensorShape - Should be {point dimension count, point count, 1}

      step - Smoothing value, higher value meaning higher smoothness

      outTensor - Float32Array of size (point dimension count * point count). Smoothed points are written to this array

      Parameters

      • inTensor: Float32Array
      • tensorShape: vec3
      • step: number
      • outTensor: Float32Array

      Returns void

    • Applies softArgMax function to the inTensor, with the specified inShape. The result is put into outTensor.

      If inShape = {width, height, channels}, then the shape of outTensor should be {1, 2, channels}.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • outTensor: Float32Array
      • normalized: boolean

      Returns void

    • Applies softMax function to inTensor, with the specified inShape. The result is put into outTensor.

      If inShape = {width, height, channels}, then the shape of outTensor should be {1, 1, channels}.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • outTensor: Float32Array

      Returns void

    • Estimates the object pose given a set of object points (inObjectPoints), their corresponding image projections (inImagePoints), and the intrinsic camera matrix (transform).

      inObjectPoints - Float32Array of size (point count * 3). 3D points of the object

      inImagePoints - Float32Array of size (point count * 2). 2D points which are projections of the points in inObjectPoints, with some applied transformation we are attempting to find

      imagePointsShape - Should be {2, point count, 1}

      transform - Intrinsic camera matrix

      flags - Currently unused, can be left as 0. Further functionality may be added in the future.

      outRotTrans - Float32Array of size 6 where results are written. Describes object transformation:

      vec3(outRotTrans[0], outRotTrans[1], outRotTrans[2])* - object rotation in Rodrigues format

      vec3(outRotTrans[3], outRotTrans[4], outRotTrans[5])* - object position in 3D space

      Parameters

      • inObjectPoints: Float32Array
      • inImagePoints: Float32Array
      • imagePointsShape: vec3
      • transform: mat3
      • flags: number
      • outRotTrans: Float32Array

      Returns boolean

    • Estimates the object pose given a set of object points (inObjectPoints), their corresponding image projections (inImagePoints), and the intrinsic camera matrix (cameraIntrinsicsMatrix). Similar to solvePnP but has additional camera distortion coefficient settings.

      distortionCoeff - Input vector of distortion coefficients. If the vector is null - the zero distortion coefficients are assumed.

      distortionCoeffShape - {numElements, 1, 1} where numElements can be 4, 5, 8 or 12.

      useExtrinsicGuess - The function uses the provided outRotTrans values as initial approximations of the rotation and translation vectors and further optimizes them.

      Parameters

      • inObjectPoints: Float32Array
      • inImagePoints: Float32Array
      • imagePointsShape: vec3
      • cameraIntrinsicsMatrix: mat3
      • distortionCoeff: Float32Array
      • distortionCoeffShape: vec3
      • useExtrinsicGuess: boolean
      • flags: number
      • outRotTrans: Float32Array

      Returns boolean

    • solvePnPRansac is very similar to solvePnPExtended except that it uses Random Sample Consensus ( RANSAC ) for robustly estimating the pose.

      iterationsCount - The number of times the minimum number of points are picked and the parameters estimated

      reprojectionError - As mentioned earlier in RANSAC the points for which the predictions are close enough are called inliers. This parameter value is the maximum allowed distance between the observed and computed point projections to consider it an inlier.

      confidence - Number of inliers. If the algorithm at some stage finds more inliers than confidence, it finishes. outInliers - Output array that contains indices of inliers in objectPoints and imagePoints . outRotTrans

      Parameters

      • inObjectPoints: Float32Array
      • inImagePoints: Float32Array
      • imagePointsShape: vec3
      • cameraIntrinsicsMatrix: mat3
      • distortionCoeff: Float32Array
      • distortionCoeffShape: vec3
      • useExtrinsicGuess: boolean
      • iterationsCount: number
      • reprojectionError: number
      • confidence: number
      • flags: number
      • outInliers: Uint8Array
      • outRotTrans: Float32Array

      Returns boolean

    • Applies subpixelArgMax function to the inTensor, with the specified inShape and kernel size. The result is put into outTensor.

      If inShape = {width, height, channels}, then the shape of outTensor should be {1, 2, channels}.

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • outTensor: Float32Array
      • kernelSize: number

      Returns void

    • Subtracts inTensorB from inTensorA and puts the result into outTensor. See the broadcasting rules for elementwise operations.

      Parameters

      • inTensorA: Float32Array
      • inShapeA: vec3
      • inTensorB: Float32Array
      • inShapeB: vec3
      • outTensor: Float32Array

      Returns void

    • Calculate the sum of the inTensor elements and store result in the outTensor.

      Axis specifies axis along which a sum is performed., e.g:

      axis(0, 0, 0): the sum will be performed on the whole tensor

      axis(0, 0, 1): the sum will be performed along the z axis. outTensor will store inShape.z values

      axis(1, 1, 0): the sum will be performed along x and y axes. outTensor will store

      inShape.x * inShape.y values, where outTensor[y][x] is the sum of all inTensor[0..inShape.z-1][y][x] values

      Parameters

      • inTensor: Float32Array
      • inShape: vec3
      • axis: vec3
      • outTensor: Float32Array

      Returns void

    • Exposes User Data

      Converts the texture to a set of 0-255 grayscale values, and outputs the result into outTensor.

      outTensor should be a Uint8Array of shape {width, height, 1}.

      Parameters

      • texture: Texture
      • grayscaleBuffer: Uint8Array
      • grayscaleBufferShape: vec3

      Returns void

    MMNEPVFCICPMFPCPTTAAATR