--- title: Capacitor ML Kit Subject Segmentation Plugin for Android description: Unofficial Capacitor plugin for Google ML Kit Subject Segmentation SDK to segment subjects in images. tags: - Android search: boost: 2 faq: true github_repo: capawesome-team/capacitor-mlkit npm_package: "@capacitor-mlkit/subject-segmentation" --- # Capacitor ML Kit Subject Segmentation Plugin Unofficial Capacitor plugin for [ML Kit Subject Segmentation](https://developers.google.com/ml-kit/vision/subject-segmentation).[^1]
Deliver Live Updates to your Capacitor app with Capawesome Cloud
## Use Cases The Subject Segmentation plugin is typically used to separate the subject of a picture from its background, for example: - **Background removal**: Remove or replace the background of a photo, for example for product images or portraits. - **Photo editing**: Build editing features that combine the segmented image with new backgrounds or effects. - **Sticker creation**: Turn photos into cutouts or stickers that users can share in chats and posts. ## Compatibility | Plugin Version | Capacitor Version | Status | | -------------- | ----------------- | -------------- | | 8.x.x | >=8.x.x | Active support | ## Installation You can use our **AI-Assisted Setup** to install the plugin. Add the [Capawesome Skills](https://github.com/capawesome-team/skills) to your AI tool using the following command: ```bash npx skills add capawesome-team/skills --skill capacitor-plugins ``` Then use the following prompt: ``` Use the `capacitor-plugins` skill from `capawesome-team/skills` to install the `@capacitor-mlkit/subject-segmentation` plugin in my project. ``` If you prefer **Manual Setup**, install the plugin by running the following commands and follow the platform-specific instructions below: ```bash npm install @capacitor-mlkit/subject-segmentation npx cap sync ``` ### Android #### API level This plugin requires a minimum API level of 24. #### Metadata You need to add the following meta data **in** the `application` tag in your `AndroidManifest.xml`: ```xml ``` ## Usage The following example shows how to segment a subject from the background. ### Segment a subject from the background Pass the local path of an image file to `processImage(...)` to perform the segmentation. You can optionally scale the image using the `width` and `height` options and adjust the confidence threshold. The result contains the path to the segmented image file along with its width and height: ```typescript import { SubjectSegmentation } from '@capacitor-mlkit/subject-segmentation'; const processImage = async () => { const { path } = await SubjectSegmentation.processImage({ path: 'path/to/image.jpg', confidence: 0.7, }); return path; }; ``` ## API * [`processImage(...)`](#processimage) * [`isGoogleSubjectSegmentationModuleAvailable()`](#isgooglesubjectsegmentationmoduleavailable) * [`installGoogleSubjectSegmentationModule()`](#installgooglesubjectsegmentationmodule) * [`addListener('googleSubjectSegmentationModuleInstallProgress', ...)`](#addlistenergooglesubjectsegmentationmoduleinstallprogress-) * [`removeAllListeners()`](#removealllisteners) * [Interfaces](#interfaces) * [Enums](#enums) ### processImage(...) ```typescript processImage(options: ProcessImageOptions) => Promise ``` Performs segmentation on an input image. Only available on Android and iOS. | Param | Type | | ------------- | ------------------------------------------------------------------- | | **`options`** | ProcessImageOptions | **Returns:** Promise<ProcessImageResult> **Since:** 7.2.0 -------------------- ### isGoogleSubjectSegmentationModuleAvailable() ```typescript isGoogleSubjectSegmentationModuleAvailable() => Promise ``` Check if the Google Subject Segmentation module is available. If the Google Subject Segmentation module is not available, you can install it by using `installGoogleSubjectSegmentationModule()`. Only available on Android. **Returns:** Promise<IsGoogleSubjectSegmentationModuleAvailableResult> **Since:** 7.2.0 -------------------- ### installGoogleSubjectSegmentationModule() ```typescript installGoogleSubjectSegmentationModule() => Promise ``` Install the Google Subject Segmentation module. **Attention**: This only starts the installation. The `googleSubjectSegmentationModuleInstallProgress` event listener will notify you when the installation is complete. Only available on Android. **Since:** 7.2.0 -------------------- ### addListener('googleSubjectSegmentationModuleInstallProgress', ...) ```typescript addListener(eventName: 'googleSubjectSegmentationModuleInstallProgress', listenerFunc: (event: GoogleSubjectSegmentationModuleInstallProgressEvent) => void) => Promise ``` Called when the Google Subject Segmentation module is installed. Only available on Android. | Param | Type | | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------- | | **`eventName`** | 'googleSubjectSegmentationModuleInstallProgress' | | **`listenerFunc`** | (event: GoogleSubjectSegmentationModuleInstallProgressEvent) => void | **Returns:** Promise<PluginListenerHandle> **Since:** 7.2.0 -------------------- ### removeAllListeners() ```typescript removeAllListeners() => Promise ``` Remove all listeners for this plugin. Only available on Android. **Since:** 7.2.0 -------------------- ### Interfaces #### ProcessImageResult | Prop | Type | Description | Since | | ------------ | ------------------- | ------------------------------------- | ----- | | **`path`** | string | The path to the segmented image file. | 7.2.0 | | **`width`** | number | Returns the width of the image file. | 7.2.0 | | **`height`** | number | Returns the height of the image file. | 7.2.0 | #### ProcessImageOptions | Prop | Type | Description | Default | Since | | ---------------- | ------------------- | ----------------------------------------------------------------------------------------- | ---------------- | ----- | | **`path`** | string | The local path to the image file. | | 7.2.0 | | **`width`** | number | Scale the image to this width. If no `height` is given, it will respect the aspect ratio. | | 7.2.0 | | **`height`** | number | Scale the image to this height. If no `width` is given, it will respect the aspect ratio. | | 7.2.0 | | **`confidence`** | number | Sets the confidence threshold. | 0.9 | 7.2.0 | #### IsGoogleSubjectSegmentationModuleAvailableResult | Prop | Type | Description | Since | | --------------- | -------------------- | ------------------------------------------------------------------- | ----- | | **`available`** | boolean | Whether or not the Google Subject Segmentation module is available. | 7.2.0 | #### PluginListenerHandle | Prop | Type | | ------------ | ----------------------------------------- | | **`remove`** | () => Promise<void> | #### GoogleSubjectSegmentationModuleInstallProgressEvent | Prop | Type | Description | Since | | -------------- | ------------------------------------------------------------------------------------------------------------------- | -------------------------------------------------------------- | ----- | | **`state`** | GoogleSubjectSegmentationModuleInstallState | The current state of the installation. | 7.2.0 | | **`progress`** | number | The progress of the installation in percent between 0 and 100. | 7.2.0 | ### Enums #### GoogleSubjectSegmentationModuleInstallState | Members | Value | Since | | --------------------- | -------------- | ----- | | **`UNKNOWN`** | 0 | 7.2.0 | | **`PENDING`** | 1 | 7.2.0 | | **`DOWNLOADING`** | 2 | 7.2.0 | | **`CANCELED`** | 3 | 7.2.0 | | **`COMPLETED`** | 4 | 7.2.0 | | **`FAILED`** | 5 | 7.2.0 | | **`INSTALLING`** | 6 | 7.2.0 | | **`DOWNLOAD_PAUSED`** | 7 | 7.2.0 | ## FAQ ### What are the requirements to use this plugin on Android? The plugin requires a minimum Android API level of 24. You also need to add the `com.google.mlkit.vision.DEPENDENCIES` meta data to your `AndroidManifest.xml` as described in the [Installation](#installation) section. ### Why do I need the Google Subject Segmentation module on Android? On Android, the segmentation is performed by the Google Subject Segmentation module. Use the `isGoogleSubjectSegmentationModuleAvailable()` method to check if the module is available on the device and the `installGoogleSubjectSegmentationModule()` method to install it if it is not. ### How do I know when the module installation is complete? The `installGoogleSubjectSegmentationModule()` method only starts the installation. Add a listener for the `googleSubjectSegmentationModuleInstallProgress` event to get notified about the current state of the installation and its progress in percent. ### How can I adjust the accuracy of the segmentation? You can set the confidence threshold using the `confidence` option of the `processImage(...)` method. The default value is `0.9`. Experiment with different values to find the best result for your images. ### How can I scale the image before it is processed? Use the `width` and `height` options of the `processImage(...)` method to scale the image. If only one of the two values is given, the aspect ratio of the image is respected. ### Can I use this plugin with Ionic, React, Vue or Angular? Yes, the plugin is framework-agnostic. It works in any Capacitor app regardless of the web framework, including Ionic with Angular, React, or Vue, as well as plain JavaScript projects. ## Related Plugins - [Selfie Segmentation](https://capawesome.io/docs/sdks/capacitor/mlkit/selfie-segmentation/): Unofficial Capacitor plugin for ML Kit Selfie Segmentation. - [Face Detection](https://capawesome.io/docs/sdks/capacitor/mlkit/face-detection/): Unofficial Capacitor plugin for ML Kit Face Detection. - [Face Mesh Detection](https://capawesome.io/docs/sdks/capacitor/mlkit/face-mesh-detection/): Unofficial Capacitor plugin for ML Kit Face Mesh Detection. ## Terms & Privacy This plugin uses the [Google ML Kit](https://developers.google.com/ml-kit): - [Terms & Privacy](https://developers.google.com/ml-kit/terms) - [Android Data Disclosure](https://developers.google.com/ml-kit/android-data-disclosure) - [iOS Data Disclosure](https://developers.google.com/ml-kit/ios-data-disclosure) ## Newsletter Stay up to date with the latest news and updates about the Capawesome, Capacitor, and Ionic ecosystem by subscribing to our [Capawesome Newsletter](https://cloud.capawesome.io/newsletter/). ## Changelog See [CHANGELOG.md](https://github.com/capawesome-team/capacitor-mlkit/blob/main/packages/subject-segmentation/CHANGELOG.md). ## License See [LICENSE](https://github.com/capawesome-team/capacitor-mlkit/blob/main/packages/subject-segmentation/LICENSE). [^1]: This project is not affiliated with, endorsed by, sponsored by, or approved by Google LLC or any of their affiliates or subsidiaries.