Image-Based Testing

Image-Based Testing

How do you test mobile apps if you don’t have direct access to its objects? If you’re testing iOS, have a seat, enjoy some coffee. Better yet, watch for the upcoming blog on instrumenting iOS mobile apps. You must have an instrumented app to test iOS mobile (for other than web based testing). If you’re testing Android apps, you have another tool in the arsenal: image-based testing using image sets. Image sets are collections of images that correspond to objects on the screen. A simple example: if I want to click the Phone icon, it has a different graphic on the standard Samsung desktop vs. a replacement desktop launcher with its own set of icons. Using image sets, I can recognize the phone icon in both environments.

Image sets are your friend when:

1. There are multiple editions of the app that have different UIs.
2. You want to interact with the desktop. The desktop launcher app can’t be instrumented without a signed package, so here you can use image based recognition.
3. You want to manipulate the system controlled notification dropdown or settings screens. This is really the same as #2. You can’t instrument the OS, so you need image recognition instead.

TestComplete stores images in a tree structure:

• Image Repository
  • Image Set
    • Items
      • Image Strip

The Image Repository project item holds Image Sets, typically one per screen. The Image Set Items represent on-screen objects, for example, the “Phone” icon. The Image Strip holds multiple images for each item, one for each visual representation. For example, the “Phone” icon as displayed on different devices, resolutions and themes.

To get started, add a Image Repository project item to your project if it doesn’t already exist. Then right-click the Image Repository and select Add > New Item… Give the Image Set a name that represents the screen you want to work with and click OK. In this case, I’m working with the desktop launcher app and its icons:

Next, click in the Items list and select Add Item… to grab the initial image. TestComplete will take a little time to gather UI information from the device. The time it takes will depend on your PC, smart device, the USB protocol, even the USB cable that transfers the data.

In the Select Object from Screen dialog, select an area by dragging upper-left-to-lower-right. Then click the Select button.

Give the image a name and click the Finish button. The name, e.g. “Phone” in the Items list on the left hand refers to one on-screen object. The bottom of the screen shows a sample Script Line call to the Touch() method.

When you select an item in the left-hand Items pane, you get a Preview, Parameters and the Image Strip along the bottom. There’s only one image now, but each item in the Image Strip will represent the Phone item. The parameters for each element in the Image Strip let you tweak how the image is located.

To check that your image recognition parameters are OK, right-click the item and select Highlight on Screen… To paraphrase Will Smith, TestComplete will “flashy-red-thing” around the recognized area.

Once the on-screen object is found, you can reference it in script. Starting with the TestComplete ImageRepository object, reference the image set (“DesktopIcons” in this case) and finally the “Phone” image set item. You can check if the item exists, touch the item or use the item in checkpoints for verification.

if (ImageRepository.DesktopIcons.Phone.Exists()){
  ImageRepository.DesktopIcons.Phone.CheckImage(); 
  ImageRepository.DesktopIcons.Phone.Touch(); 
}

Wrap up

We implemented image-based testing by creating an image set for a phone icon, then called its script methods. In the next installment we’ll add multiple images to handle color and pixel variations. See you tomorrow.

The post Image-Based Testing appeared first on Falafel Software Blog.