Introducing the ImagePicker & Image API

Hero banner

Overview

The latest release of the Fitbit Mobile application added two great new features that allow developers to easily build apps and clock faces which use images directly from a user's mobile phone, or appropriately licensed images downloaded from the internet.

Although developers had previously found some workarounds for dynamically loading images onto a device, these new features now make the whole process painless.

We'll begin by taking a look at the ImagePicker component, then see how the Image API is used to convert images so they're suitable for the device.

The ImagePicker Component

The new ImagePicker component within the Settings API lets users select, crop, and scale a photo from their phone's photo library within their application's settings page. The component is initialized with familiar parameters, plus some additional ones which determine the dimensions of the output image.

Usage

In the following example we will initiate the ImagePicker so it will persist a photo within settingsStorage with a key of background-image. We need this key to retrieve the image data within the companion in order to send the image to the device.

We want to return a full-screen image and we may be dealing with multiple device screen sizes, so we need to dynamically populate the imageWidth and imageHeight values. We can use the Peer API in the companion to persist the screen dimensions into settingsStorage, so we can use those values within the settings page. See the Multiple Devices guide for further information.

Step 1

Persist the device screen dimensions into settingsStorage:

import { device } from "peer";
import { settingsStorage } from "settings";

settingsStorage.setItem("screenWidth", device.screen.width);
settingsStorage.setItem("screenHeight", device.screen.height);

Step 2

Render a settings page which lets the user select, scale, and crop a photo. The ImagePicker will use the imageWidth and imageHeight properties to ensure that the image file will be returned in the correct size, and aspect ratio.

function mySettings(props) {
  let screenWidth = props.settingsStorage.getItem("screenWidth");
  let screenHeight = props.settingsStorage.getItem("screenHeight");

  return (
    <Page>
        <ImagePicker
          title="Background Image"
          description="Pick an image to use as your background."
          label="Pick a Background Image"
          sublabel="Background image picker"
          settingsKey="background-image"
          imageWidth={ screenWidth }
          imageHeight={ screenHeight }
        />
    </Page>
  );
}

registerSettingsPage(mySettings);

Step 3

When the user selects their image, the settingsStorage.onchange event will be emitted to notify the companion that a new image is available.

import { settingsStorage } from "settings";

settingsStorage.onchange = function(evt) {
  if (evt.key === "background-image") {
    let imageData = JSON.parse(evt.newValue);
    // We now have our image data in: imageData.imageUri
  }
}

Step 4

The ImagePicker component returns all images in Base64 encoded image/png Data URI format. This is great, because it means we're not losing any image quality, but unfortunately Fitbit OS devices don't natively support png format images.

We will need to use the Image API to convert our image before sending it to the device.

The Image API

The Image API can be used to convert a source image into a format that is supported by Fitbit OS. The API is Promise based, so it's simple to chain each of the required steps to produce an image.

The source image can either be an image Data URI (e.g. data:image/png;base64,/9j/4AAQSkZJRgABAQEAYABgAAD...), which the ImagePicker provides by default, or it can be an ArrayBuffer, which fetch() can easily provide.

The means the Image API can easily convert images from the ImagePicker, or appropriately licensed images downloaded from the internet using fetch().

Progressive JPEG files cannot be rendered on the device, but you don't need to worry as the Image API will always output JPEG files that are compatible with the device.

Output Formats

Fitbit OS devices currently supports two image formats: jpeg, and a proprietary format called txi.

Each format has different advantages and disadvantages:

txi files have hardware support, so they are fastest to render on the device, but the file size can be relatively large, so they take longer to transfer from the companion to the device. If you need transparency in your image, you should use this format.

jpeg files are slightly slower to render on the device, but support compression. Because the file size is smaller, it's therefore quicker to transfer, but the compression can negatively affect overall image quality.

Both formats are fully supported, and the developer can choose which is best for their intended usage.

One approach which tends to work well is to convert the source image into a jpeg with medium compression (40-50), then use the JPEG API on the device to save the jpeg as a txi file. With this method, you get the benefit of reduced transfer duration, and improved render performance on the device. You can see an example of this approach in the SDK Image Clock example clock face.

Source to JPG Conversion

Let's take a look at how the png image from the ImagePicker can be converted into a jpeg, and queued for File Transfer to the device.

Firstly we pass the imageUri from the ImagePicker into the Image constructor. Then we export the image and we instruct the Image API to replace transparent pixels with white, and use JPEG compression 40. Finally we add the file into the Outbox queue for transfer.

import { outbox } from "file-transfer";
import { Image } from "image";

settingsStorage.onchange = function(evt) {
  if (evt.key === "background-image") {
    compressAndTransferImage(evt.newValue);
  }
};

function compressAndTransferImage(settingsValue) {
  const imageData = JSON.parse(settingsValue);
  Image.from(imageData.imageUri)
    .then(image =>
      image.export("image/jpeg", {
        background: "#FFFFFF",
        quality: 40
      })
    )
    .then(buffer => outbox.enqueue(`${Date.now()}.jpg`, buffer))
    .then(fileTransfer => {
      console.log(`Enqueued ${fileTransfer.name}`);
    });
}

Check out the SDK Photo Picker example clock face for a complete implementation.

Source to TXI Conversion

In this example we will fetch a jpeg from the internet, the convert it into a txi file.

Note that when exporting a txi file, we can specify a background color for transparent pixels, but not compression quality.

import { outbox } from "file-transfer";
import { Image } from "image";

fetch("https://.../some-file.png")
  .then(response => response.arrayBuffer())
  .then(buffer => Image.from(buffer, "image/png"))
  .then(image =>
    image.export("image/vnd.fitbit.txi", {
      background: "#FFFFFF"
    })
  )
  .then(buffer => outbox.enqueue(`${Date.now()}.jpg`, buffer))
  .then(fileTransfer => {
    console.log(`Enqueued ${fileTransfer.name}`);
  });

Check out the SDK Image Clock example for a complete implementation, including how to convert the jpeg into txi format on the device using the JPEG API.

Until Next Time

Follow @fitbitdev on Twitter, join our Fitbit Community Forum, or get news straight to your inbox by signing up below. Curious to see the amazing work Fitbit Developers have done so far? Keep tabs on the #Made4Fitbit Twitter hashtag.