--- description: Unofficial Capacitor plugin for the Google ML Kit SDK to detect, track, and classify the prominent objects in images on Android and iOS. title: Capacitor ML Kit Object Detection Plugin - Capawesome image: https://capawesome.io/docs/assets/images/social/sdks/capacitor/mlkit/object-detection.png --- [Skip to content ](#capacitor-ml-kit-object-detection-plugin) [πŸ” Introducing the **Capacitor Vault** plugin β€” store secrets behind biometrics or a device passcode. ](/blog/announcing-the-capacitor-vault-plugin/) * [ SDKs ](/docs/sdks/) * [ Formbricks ](/docs/sdks/capacitor/formbricks/) * [ Geocoder ](/docs/sdks/capacitor/geocoder/) * [ Google Sign-In ](/docs/sdks/capacitor/google-sign-in/) * [ Grafana Faro ](/docs/sdks/capacitor/grafana-faro/) * [ Gyroscope ](/docs/sdks/capacitor/gyroscope/) * [ Haptics ](/docs/sdks/capacitor/haptics/) * [ Home Indicator ](/docs/sdks/capacitor/home-indicator/) * [ In-App Browser ](/docs/sdks/capacitor/in-app-browser/) * [ Install Referrer ](/docs/sdks/capacitor/install-referrer/) * [ Keep Awake ](/docs/sdks/capacitor/keep-awake/) * [ libSQL ](/docs/sdks/capacitor/libsql/) * [ Light Sensor ](/docs/sdks/capacitor/light-sensor/) * [ Live Update ](/docs/sdks/capacitor/live-update/) * [ Localization ](/docs/sdks/capacitor/localization/) * [ Mail Composer ](/docs/sdks/capacitor/mail-composer/) * [ Managed Configurations ](/docs/sdks/capacitor/managed-configurations/) * [ Maps Launcher ](/docs/sdks/capacitor/maps-launcher/) * [ Media Session ](/docs/sdks/capacitor/media-session/) * [ ML Kit ](/docs/sdks/capacitor/mlkit/) * [ iOS ](#ios) * [ Configuration ](#configuration) * [ Demo ](#demo) * [ Usage ](#usage) * [ API ](#api) * [ FAQ ](#faq) * [ Related Plugins ](#related-plugins) * [ Terms & Privacy ](#terms-privacy) * [ Newsletter ](#newsletter) * [ Changelog ](#changelog) * [ License ](#license) * [ Pose Detection ](/docs/sdks/capacitor/mlkit/pose-detection/) * [ Selfie Segmentation ](/docs/sdks/capacitor/mlkit/selfie-segmentation/) * [ Smart Reply ](/docs/sdks/capacitor/mlkit/smart-reply/) * [ Subject Segmentation ](/docs/sdks/capacitor/mlkit/subject-segmentation/) * [ Text Recognition ](/docs/sdks/capacitor/mlkit/text-recognition/) * [ Translation ](/docs/sdks/capacitor/mlkit/translation/) * [ Navigation Bar ](/docs/sdks/capacitor/navigation-bar/) * [ Network ](/docs/sdks/capacitor/network/) * [ NFC ](/docs/sdks/capacitor/nfc/) * [ Node.js ](/docs/sdks/capacitor/nodejs/) * [ OAuth ](/docs/sdks/capacitor/oauth/) * [ Passkeys ](/docs/sdks/capacitor/passkeys/) * [ Password Autofill ](/docs/sdks/capacitor/password-autofill/) * [ PDF Generator ](/docs/sdks/capacitor/pdf-generator/) * [ PDF Viewer ](/docs/sdks/capacitor/pdf-viewer/) * [ Pedometer ](/docs/sdks/capacitor/pedometer/) * [ Permissions ](/docs/sdks/capacitor/permissions/) * [ Phone Dialer ](/docs/sdks/capacitor/phone-dialer/) * [ Photo Editor ](/docs/sdks/capacitor/photo-editor/) * [ Photo Manipulator ](/docs/sdks/capacitor/photo-manipulator/) * [ PixLive ](/docs/sdks/capacitor/pixlive/) * [ PostHog ](/docs/sdks/capacitor/posthog/) * [ Printer ](/docs/sdks/capacitor/printer/) * [ Privacy Screen ](/docs/sdks/capacitor/privacy-screen/) * [ Proximity Sensor ](/docs/sdks/capacitor/proximity-sensor/) * [ Purchases ](/docs/sdks/capacitor/purchases/) * [ RealtimeKit ](/docs/sdks/capacitor/realtimekit/) * [ Root Detection ](/docs/sdks/capacitor/root-detection/) * [ Screen Brightness ](/docs/sdks/capacitor/screen-brightness/) * [ Screen Orientation ](/docs/sdks/capacitor/screen-orientation/) * [ Screen Reader ](/docs/sdks/capacitor/screen-reader/) * [ Screenshot ](/docs/sdks/capacitor/screenshot/) * [ Secure Preferences ](/docs/sdks/capacitor/secure-preferences/) * [ Settings Launcher ](/docs/sdks/capacitor/settings-launcher/) * [ Shake ](/docs/sdks/capacitor/shake/) * [ Silent Mode ](/docs/sdks/capacitor/silent-mode/) * [ SIM ](/docs/sdks/capacitor/sim/) * [ SMS Composer ](/docs/sdks/capacitor/sms-composer/) * [ Speech Recognition ](/docs/sdks/capacitor/speech-recognition/) * [ Speech Synthesis ](/docs/sdks/capacitor/speech-synthesis/) * [ Share Target ](/docs/sdks/capacitor/share-target/) * [ Square Mobile Payments ](/docs/sdks/capacitor/square-mobile-payments/) * [ SQLite ](/docs/sdks/capacitor/sqlite/) * [ Superwall ](/docs/sdks/capacitor/superwall/) * [ System WebView ](/docs/sdks/capacitor/system-webview/) * [ Text Interaction ](/docs/sdks/capacitor/text-interaction/) * [ Text Zoom ](/docs/sdks/capacitor/text-zoom/) * [ Thermal State ](/docs/sdks/capacitor/thermal-state/) * [ Toast ](/docs/sdks/capacitor/toast/) * [ Torch ](/docs/sdks/capacitor/torch/) * [ Vault ](/docs/sdks/capacitor/vault/) * [ Volume ](/docs/sdks/capacitor/volume/) * [ Wallet ](/docs/sdks/capacitor/wallet/) * [ Wifi ](/docs/sdks/capacitor/wifi/) * [ Zip ](/docs/sdks/capacitor/zip/) * [ Cordova ](/docs/sdks/cordova/) * [ Cloud ](/docs/cloud/) * [ Integrations ](/docs/cloud/live-updates/integrations/) * Concepts * Reference * [ Troubleshooting ](/docs/cloud/live-updates/troubleshooting/) * [ FAQ ](/docs/cloud/live-updates/faq/) * [ Native Builds ](/docs/cloud/native-builds/) * [ Set Up Environments ](/docs/cloud/native-builds/environments/) * [ Overwrite Native Configurations ](/docs/cloud/native-builds/native-configurations/) * [ Auto-Increment Build Numbers ](/docs/cloud/native-builds/auto-incrementing-build-numbers/) * [ Configure the Web Build Script ](/docs/cloud/native-builds/web-build-script/) * [ Build from a Monorepo ](/docs/cloud/native-builds/monorepo/) * [ Use pnpm, Yarn, or bun ](/docs/cloud/native-builds/package-managers/) * [ Install Private npm Packages ](/docs/cloud/native-builds/npm-private-registry/) * [ Override the Java Version ](/docs/cloud/native-builds/override-java-version/) * [ Custom iOS Provisioning Profiles ](/docs/cloud/native-builds/custom-ios-provisioning-profiles/) * [ Build without Git ](/docs/cloud/native-builds/build-without-git/) * [ Access Git Behind a Firewall ](/docs/cloud/native-builds/firewall-access/) * [ Integrations ](/docs/cloud/native-builds/integrations/) * Reference * [ Troubleshooting ](/docs/cloud/native-builds/troubleshooting/) * [ FAQ ](/docs/cloud/native-builds/faq/) * [ App Store Publishing ](/docs/cloud/app-store-publishing/) * [ Submit a Build ](/docs/cloud/app-store-publishing/submit-a-build/) * [ Submit Automatically After a Build ](/docs/cloud/app-store-publishing/submit-automatically/) * [ Troubleshooting ](/docs/cloud/app-store-publishing/troubleshooting/) * [ FAQ ](/docs/cloud/app-store-publishing/faq/) * [ Automations ](/docs/cloud/automations/) * [ Reference ](/docs/cloud/automations/reference/) * [ Troubleshooting ](/docs/cloud/automations/troubleshooting/) * [ FAQ ](/docs/cloud/automations/faq/) * [ Assist ](/docs/cloud/assist/) * [ CLI ](/docs/cloud/cli/) * APIs and SDKs * [ Webhooks ](/docs/cloud/webhooks/) * [ Integrations ](/docs/cloud/integrations/) * Account * [ Organization ](/docs/cloud/organizations/) * [ Two-Factor Enforcement ](/docs/cloud/organizations/two-factor-authentication/) * [ Audit Logs ](/docs/cloud/organizations/audit-logs/) * [ Billing ](/docs/cloud/organizations/billing/) * [ License Keys ](/docs/cloud/license-keys/) * [ AI ](/docs/ai/) * [ Insiders ](/docs/insiders/) * [ Billing & Plans ](/docs/insiders/billing-and-plans/) * [ FAQ ](/docs/insiders/faq/) * [ License ](https://capawesome.io/legal/eula/) * [ Support ](/docs/support/) * [ Contributing ](/docs/contributing/) * Contributing code * [ Code of Conduct ](/docs/contributing/code-of-conduct/) * [ Questions ](https://docs.github.com/en/discussions/collaborating-with-your-community-using-discussions/participating-in-a-discussion#creating-a-discussion) * [ Blog ](/blog/) * Categories * [ iOS ](#ios) * [ Configuration ](#configuration) * [ Demo ](#demo) * [ Usage ](#usage) * [ API ](#api) * [ FAQ ](#faq) * [ Related Plugins ](#related-plugins) * [ Terms & Privacy ](#terms-privacy) * [ Newsletter ](#newsletter) * [ Changelog ](#changelog) * [ License ](#license) # Capacitor ML Kit Object Detection Plugin[ΒΆ](#capacitor-ml-kit-object-detection-plugin "Permanent link") Unofficial Capacitor plugin for [ML Kit Object Detection and Tracking](https://developers.google.com/ml-kit/vision/object-detection).[1](#fn:1) [ ![Deliver Live Updates to your Capacitor app with Capawesome Cloud](../../../../assets/external/cloud.capawesome.io/assets/banners/cloud-build-and-deploy-capacitor-apps.69628c3f.png) ](https://cloud.capawesome.io/) ## Use Cases[ΒΆ](#use-cases "Permanent link") The Object Detection plugin is typically used whenever an app needs to locate and classify the prominent objects in an image, for example: * **Visual search**: Let users draw a box around an object in a photo and search for similar products. * **Content categorization**: Automatically tag images by the objects they contain (fashion goods, food, plants, and more). * **Image cropping**: Use the bounding box of the most prominent object to automatically crop or focus an image. * **Inventory management**: Detect and count objects in a photo to support cataloging workflows. ## Compatibility[ΒΆ](#compatibility "Permanent link") | Plugin Version | Capacitor Version | Status | | -------------- | ----------------- | -------------- | | 8.x.x | \>=8.x.x | Active support | ## Installation[ΒΆ](#installation "Permanent link") You can use our **AI-Assisted Setup** to install the plugin. Add the [Capawesome Skills](https://github.com/capawesome-team/skills) to your AI tool using the following command: `[](#%5F%5Fcodelineno-0-1)npx skills add capawesome-team/skills --skill capacitor-plugins ` Then use the following prompt: `` [](#%5F%5Fcodelineno-1-1)Use the `capacitor-plugins` skill from `capawesome-team/skills` to install the `@capacitor-mlkit/object-detection` plugin in my project. `` If you prefer **Manual Setup**, install the plugin by running the following commands and follow the platform-specific instructions below: `[](#%5F%5Fcodelineno-2-1)npm install @capacitor-mlkit/object-detection [](#%5F%5Fcodelineno-2-2)npx cap sync ` **Attention**: This plugin **only supports CocoaPods** for iOS dependency management. Swift Package Manager (SPM) is not supported for the ML Kit SDK, see [this comment](https://github.com/googlesamples/mlkit/issues/180#issuecomment-1298964099). ### Android[ΒΆ](#android "Permanent link") #### Variables[ΒΆ](#variables "Permanent link") If needed, you can define the following project variable in your app’s `variables.gradle` file to change the default version of the dependency: * `$mlkitObjectDetectionVersion` version of `com.google.mlkit:object-detection` (default: `17.0.2`) This can be useful if you encounter dependency conflicts with other plugins in your project. ### iOS[ΒΆ](#ios "Permanent link") #### Minimum Deployment Target[ΒΆ](#minimum-deployment-target "Permanent link") Make sure to set the deployment target in your `ios/App/Podfile` to at least `15.5`: `[](#%5F%5Fcodelineno-3-1)platform :ios, '15.5' ` ## Configuration[ΒΆ](#configuration "Permanent link") No configuration required for this plugin. ## Demo[ΒΆ](#demo "Permanent link") A working example can be found here: [robingenz/capacitor-mlkit-plugin-demo](https://github.com/robingenz/capacitor-mlkit-plugin-demo) ## Usage[ΒΆ](#usage "Permanent link") The following example shows how to detect objects in an image. ### Detect objects in an image[ΒΆ](#detect-objects-in-an-image "Permanent link") Detect objects in an image at a local path. You can optionally enable classification and the detection of multiple objects. Only available on Android and iOS: `[](#%5F%5Fcodelineno-4-1)import { ObjectDetection } from '@capacitor-mlkit/object-detection'; [](#%5F%5Fcodelineno-4-2) [](#%5F%5Fcodelineno-4-3)const processImage = async () => { [](#%5F%5Fcodelineno-4-4) const { detectedObjects } = await ObjectDetection.processImage({ [](#%5F%5Fcodelineno-4-5) path: 'path/to/image.jpg', [](#%5F%5Fcodelineno-4-6) shouldEnableClassification: true, [](#%5F%5Fcodelineno-4-7) shouldEnableMultipleObjects: true, [](#%5F%5Fcodelineno-4-8) }); [](#%5F%5Fcodelineno-4-9) return detectedObjects; [](#%5F%5Fcodelineno-4-10)}; ` ## API[ΒΆ](#api "Permanent link") * [processImage(...)](#processimage) * [Interfaces](#interfaces) ### processImage(...)[ΒΆ](#processimage "Permanent link") `[](#%5F%5Fcodelineno-5-1)processImage(options: ProcessImageOptions) => Promise ` Detects objects in the supplied image. Each detected object is localized with a bounding box and, if classification is enabled, labeled with one of five coarse categories. Only available on Android and iOS. | Param | Type | | ----------- | ------------------------------------------- | | **options** | [ProcessImageOptions](#processimageoptions) | **Returns:** `Promise<[ProcessImageResult](#processimageresult)>` **Since:** 8.2.0 --- ### Interfaces[ΒΆ](#interfaces "Permanent link") #### ProcessImageResult[ΒΆ](#processimageresult "Permanent link") | Prop | Type | Description | Since | | ------------------- | ------------------ | --------------------- | ----- | | **detectedObjects** | DetectedObject\[\] | The detected objects. | 8.2.0 | #### DetectedObject[ΒΆ](#detectedobject "Permanent link") Represents an object detected in an image. | Prop | Type | Description | Since | | --------------- | --------------- | ---------------------------------------------------------------------------------------------------- | ----- | | **boundingBox** | [Rect](#rect) | The bounding box of the detected object. | 8.2.0 | | **trackingId** | number | The tracking identifier of the detected object. Only set when the object is detected in stream mode. | 8.2.0 | | **labels** | ObjectLabel\[\] | The labels of the detected object. The list is empty if classification is disabled. | 8.2.0 | #### Rect[ΒΆ](#rect "Permanent link") Represents a rectangle. | Prop | Type | Description | Since | | ---------- | ------ | ------------------------------------------------- | ----- | | **left** | number | The left coordinate of the rectangle in pixels. | 8.2.0 | | **top** | number | The top coordinate of the rectangle in pixels. | 8.2.0 | | **right** | number | The right coordinate of the rectangle in pixels. | 8.2.0 | | **bottom** | number | The bottom coordinate of the rectangle in pixels. | 8.2.0 | #### ObjectLabel[ΒΆ](#objectlabel "Permanent link") Represents a label of a detected object. | Prop | Type | Description | Since | | -------------- | ------ | ----------------------------------------------------------------- | ----- | | **index** | number | The index of the label. | 8.2.0 | | **text** | string | The text of the label (e.g. Food). The text is always in English. | 8.2.0 | | **confidence** | number | The confidence of the label. The value is between 0.0 and 1.0. | 8.2.0 | #### ProcessImageOptions[ΒΆ](#processimageoptions "Permanent link") | Prop | Type | Description | Default | Since | | ------------------------------- | ------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------- | ----- | | **path** | string | The local path to the image file. | | 8.2.0 | | **shouldEnableClassification** | boolean | Whether to classify the detected objects into coarse categories. Classification returns only five coarse categories (Fashion good, Food, Home good, Place and Plant). | false | 8.2.0 | | **shouldEnableMultipleObjects** | boolean | Whether to detect multiple objects instead of only the most prominent one. When enabled, up to five objects are detected. | false | 8.2.0 | ## FAQ[ΒΆ](#faq "Permanent link") ### Which platforms are supported by this plugin?[ΒΆ](#which-platforms-are-supported-by-this-plugin "Permanent link") The `processImage(...)` method is only available on Android and iOS. The Web platform is not supported by the underlying ML Kit Object Detection SDK. ### Which model does the plugin use?[ΒΆ](#which-model-does-the-plugin-use "Permanent link") The plugin uses the base object detection and tracking model, which is bundled with the app. Custom TensorFlow Lite classifier models are currently not supported. ### Which categories does classification return?[ΒΆ](#which-categories-does-classification-return "Permanent link") When `shouldEnableClassification` is enabled, the base model classifies detected objects into only five coarse categories: `Fashion good`, `Food`, `Home good`, `Place`, and `Plant`. Any object that does not fall into one of these categories receives no label. ### Why is the `trackingId` not set?[ΒΆ](#why-is-the-trackingid-not-set "Permanent link") The plugin processes still images in single-image mode, where objects are not tracked across frames. The `trackingId` is therefore not set. It is only populated when objects are detected in stream mode, which is not supported by this plugin. ### Can I use this plugin with Ionic, React, Vue or Angular?[ΒΆ](#can-i-use-this-plugin-with-ionic-react-vue-or-angular "Permanent link") Yes, the plugin is framework-agnostic. It works in any Capacitor app regardless of the web framework, including Ionic with Angular, React, or Vue, as well as plain JavaScript projects. ## Related Plugins[ΒΆ](#related-plugins "Permanent link") * [ML Kit Barcode Scanning](https://capawesome.io/docs/sdks/capacitor/mlkit/barcode-scanning/): Scan barcodes with ML Kit Barcode Scanning. * [ML Kit Face Detection](https://capawesome.io/docs/sdks/capacitor/mlkit/face-detection/): Detect faces in images with ML Kit Face Detection. * [ML Kit Image Labeling](https://capawesome.io/docs/sdks/capacitor/mlkit/image-labeling/): Detect labels in images with ML Kit Image Labeling. ## Terms & Privacy[ΒΆ](#terms-privacy "Permanent link") This plugin uses the [Google ML Kit](https://developers.google.com/ml-kit): * [Terms & Privacy](https://developers.google.com/ml-kit/terms) * [Android Data Disclosure](https://developers.google.com/ml-kit/android-data-disclosure) * [iOS Data Disclosure](https://developers.google.com/ml-kit/ios-data-disclosure) ## Newsletter[ΒΆ](#newsletter "Permanent link") Stay up to date with the latest news and updates about the Capawesome, Capacitor, and Ionic ecosystem by subscribing to our [Capawesome Newsletter](https://cloud.capawesome.io/newsletter/). ## Changelog[ΒΆ](#changelog "Permanent link") See [CHANGELOG.md](https://github.com/capawesome-team/capacitor-mlkit/blob/main/packages/object-detection/CHANGELOG.md). ## License[ΒΆ](#license "Permanent link") See [LICENSE](https://github.com/capawesome-team/capacitor-mlkit/blob/main/packages/object-detection/LICENSE). --- 1. This project is not affiliated with, endorsed by, sponsored by, or approved by Google LLC or any of their affiliates or subsidiaries. [↩](#fnref:1 "Jump back to footnote 1 in the text") July 10, 2026 Back to top ```json {"@context": "https://schema.org", "@graph": [{"@type": "TechArticle", "@id": "https://capawesome.io/docs/sdks/capacitor/mlkit/object-detection/#article", "headline": "Capacitor ML Kit Object Detection Plugin", "name": "Capacitor ML Kit Object Detection Plugin", "description": "Unofficial Capacitor plugin for the Google ML Kit SDK to detect, track, and classify the prominent objects in images on Android and iOS.", "inLanguage": "en", "url": "https://capawesome.io/docs/sdks/capacitor/mlkit/object-detection/", "mainEntityOfPage": "https://capawesome.io/docs/sdks/capacitor/mlkit/object-detection/", "author": {"@type": "Organization", "name": "Capawesome", "url": "https://capawesome.io", "logo": {"@type": "ImageObject", "url": "https://capawesome.io/assets/images/logo.svg"}}, "publisher": {"@type": "Organization", "name": "Capawesome", "url": "https://capawesome.io", "logo": {"@type": "ImageObject", "url": "https://capawesome.io/assets/images/logo.svg"}}, "about": {"@id": "https://capawesome.io/docs/sdks/capacitor/mlkit/object-detection/#software"}}, {"@type": "SoftwareSourceCode", "@id": "https://capawesome.io/docs/sdks/capacitor/mlkit/object-detection/#software", "name": "Capacitor ML Kit Object Detection Plugin", "description": "Unofficial Capacitor plugin for the Google ML Kit SDK to detect, track, and classify the prominent objects in images on Android and iOS.", "url": "https://capawesome.io/docs/sdks/capacitor/mlkit/object-detection/", "programmingLanguage": "TypeScript", "runtimePlatform": "Capacitor", "codeRepository": "https://github.com/capawesome-team", "author": {"@type": "Organization", "name": "Capawesome", "url": "https://capawesome.io", "logo": {"@type": "ImageObject", "url": "https://capawesome.io/assets/images/logo.svg"}}, "publisher": {"@type": "Organization", "name": "Capawesome", "url": "https://capawesome.io", "logo": {"@type": "ImageObject", "url": "https://capawesome.io/assets/images/logo.svg"}}}]} {"@context": "https://schema.org", "@type": "FAQPage", "mainEntity": [{"@type": "Question", "name": "Which platforms are supported by this plugin?", "acceptedAnswer": {"@type": "Answer", "text": "The processImage(...) method is only available on Android and iOS. The Web platform is not supported by the underlying ML Kit Object Detection SDK."}}, {"@type": "Question", "name": "Which model does the plugin use?", "acceptedAnswer": {"@type": "Answer", "text": "The plugin uses the base object detection and tracking model, which is bundled with the app. Custom TensorFlow Lite classifier models are currently not supported."}}, {"@type": "Question", "name": "Which categories does classification return?", "acceptedAnswer": {"@type": "Answer", "text": "When shouldEnableClassification is enabled, the base model classifies detected objects into only five coarse categories: Fashion good, Food, Home good, Place, and Plant. Any object that does not fall into one of these categories receives no label."}}, {"@type": "Question", "name": "Why is the trackingId not set?", "acceptedAnswer": {"@type": "Answer", "text": "The plugin processes still images in single-image mode, where objects are not tracked across frames. The trackingId is therefore not set. It is only populated when objects are detected in stream mode, which is not supported by this plugin."}}, {"@type": "Question", "name": "Can I use this plugin with Ionic, React, Vue or Angular?", "acceptedAnswer": {"@type": "Answer", "text": "Yes, the plugin is framework-agnostic. It works in any Capacitor app regardless of the web framework, including Ionic with Angular, React, or Vue, as well as plain JavaScript projects."}}], "url": "https://capawesome.io/docs/sdks/capacitor/mlkit/object-detection/"} ```