Changelog

Our latest runtime, 2024.02, is now GA.

The 2024.02 runtime includes several new libraries, plus updates to Playwright and other libraries. We also removed support for Mocha.

New libraries:

• @axe-core/playwright

• @opentelemetry/api

• @opentelemetry/sdk-metrics

• @opentelemetry/exporter-metrics-otlp-grpc

• chai-json-schema

• nice-grpc

• pdftojson
  

We have also enabled the node datagram and performance modules.

With the addition of @axe-core/playwright, accessibility monitoring is now possible. Here is an introduction video on how to work with Axe and Playwright 👇

2024.02 uses Playwright 1.42.1. For a list of all library versions, see our runtime specs documentation.

The private location agent supporting 2024.02 is v3.3.0. Pull the latest version from https://hub.docker.com/r/checkly/agent

If you have questions or feedback, join our Slack community and let us know.

This patch release of the Checkly CLI fixes some small bugs and adds

Alerting additions

You can now set the alerting threshold for failing locations when using parallel scheduling. in the Web UI this configured like below 👇

For the CLI, you add the parallelRunFailureTreshold object to your alertEscalationPolicy:

alertEscalationPolicy: AlertEscalationBuilder
    .runBasedEscalation(2,
      { amount: 2, interval: 5 }, // sets reminders
      { enabled: true, percentage: 50 } // sets the parallelRunFailureThreshold
),

Global configuration fix

We missed exporting the userAgent as an option for global configuration. This option is added now. This means two things:

1. You can set a user agent in your global config, e.g.

use: {
  userAgent: 'my funky user'
}

2. Using the devices macro now sets the correct user agent header. That field was ignored earlier, e.g.

import { devices } from '@playwright/test'

use: {
   ...devices['iPhone 12 Pro Max landscape']
}

the above code will set the user agent header to:

"Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Mobile/15E148 Safari/604.1"

Learn more about the Playwright Global Configuration Beta.

To get started with the Checkly CLI just run

npm create checkly 

Or upgrade your existing installation with

npm install checkly@latest

Check the full release note on GitHub here

Parallel scheduling is now GA.

Reduce mean time to detection, get better insights when addressing outages, and get improved data granularity on all performance trends with parallel scheduling.

A check using parallel scheduling will run from each configured location each time it is scheduled, ensuring that any local outages are detected instantly. Alerts are sent as soon as a location is registered as failing or degrading, minimizing the time from failure to your team being notified.

Changing your scheduling method

With GA, parallel scheduling is now the default for all new checks and groups. To change the scheduling method, go to "Scheduling & Locations" when editing your check.

Parallel scheduling is fully supported in our monitoring as code workflow. With our CLI, just add the runParallel property to your construct:

import { ApiCheck, AssertionBuilder } from 'checkly/constructs'

new ApiCheck('hello-api-1', {
  name: 'Hello API',
  runParallel: true,
  locations: ['eu-central-1', 'us-east-1'],
  request: {
    method: 'GET',
    url: 'https://mac-demo-repo.vercel.app/api/hello',
    assertions: [
      AssertionBuilder.statusCode().equals(200)
    ],
  }
})

Similar, with Terraform just add run_parallel

resource "checkly_check" "list-all-checks" {
  name                      = "List all checks"
  type                      = "API"
  frequency                 = 0
  frequency_offset          = 10
  activated                 = false
  muted                     = false
  run_parallel              = true
  locations                 = ["eu-north-1", "eu-central-1", "us-west-1", "ap-northeast-1"]
  degraded_response_time    = 5000
  max_response_time         = 20000
  request {
    method                    = "GET"
    url                       = "https://developers.checklyhq.com/reference/getv1checks"
  }
}

Viewing check results

The check overview page will now show the progress of all locations as a check is running, giving you instant feedback if a certain location is taking longer than usual.

The check result page has been updated to support results from multiple locations. You can switch between locations in the sidebar when viewing the check result page.

Quickly compare the results from multiple locations, reviewing failures, Playwright traces or screenshots to understand how your service behaves globally.

Pricing and availability

• Parallel check scheduling is available on all current plans: Hobby, Team, and Enterprise. Deprecated plan types are not supported.

• Each location selected for a check running in parallel does a full check run per execution. An API check that runs in five locations will be billed for five check runs per execution. For more information on pricing, see our documentation: Check pricing.

Additional resources

• Global locations and scheduling documentation

• CLI documentation

• Terraform provider documentation

• If you have questions or feedback, join our Slack community and let us know.

We now support a subset of global configuration options for our Playwright-powered Browser and Multistep checks via our CLI.

To get started, upgrade your CLI to version 4.6.0 and add any supported options to the new playwrightConfig section in your checkly.config.ts file. To quickly copy the supported settings over to your checkly.config.ts file, you can run...

npx checkly sync-playwright

You should now have a checkly.config.ts file similar to the example below 👇

import { defineConfig } from 'checkly'
import { devices } from '@playwright/test'

const config = defineConfig({
  projectName: 'Checkly website',
  logicalId: 'checkly-website-project',
  checks: {
    locations: ['us-east-1', 'eu-west-1'],
    playwrightConfig: {
      timeout: 30000,
      use: {
        baseURL: 'https://www.checklyhq.com',      
        ...devices['iPhone 12 Pro Max landscape'],
        extraHTTPHeaders: {
          'x-my-custom-header': 'value',        
        }
      }
    }, 
    checkMatch: ['**/__checks__/**/*.check.ts'],
    browserChecks: {
      testMatch: ['**/__checks__/**/*.spec.ts'],
    },   
  },
})

export default config

Note that we now set the timeout, baseUrl, device properties and extraHTTPHeaders globally for all Browser and Multistep checks using the canonical Playwright configuration options. You can now write a Browser check like...

import { test, expect } from '@playwright/test'

test('Checkly pricing page', async ({ page, baseURL }) => {
  const response = await page.goto(baseURL! + '/pricing')
  expect(response?.status()).toBeLessThan(400)
})

After deploying your checks with npx checkly deploy, you can access the fully rendered configuration in the new "playwright test config"-section in the web UI editor.

This is our first step to fully support all global configuration options. We still have some dragons to slay to support globalSetup, globalTeardown, storageState and dependencies but we are working it!

Check our docs for more information.

• Checkly CLI 4.6.0 release

• Using Playwright global configuration with the CLI

• Playwright config options support matrix

The latest release of the Checkly CLI brings the following new capabilities and improvements.

Global playwright.config.ts support in Beta

We now support a subset of global configuration options for our Playwright-powered browser and multistep checks. We also have a dedicated changelog entry with all the details.

To get started, just add any supported options to the new playwrightConfig section in your checkly.config.ts file. To quickly copy the supported settings over to your checkly.config.ts file, you can run:

npx checkly sync-playwright

You should then have a config file similar to the example below 👇

import { defineConfig } from 'checkly'
import { devices } from '@playwright/test'

const config = defineConfig({
  projectName: 'Checkly website',
  logicalId: 'checkly-website-project',
  checks: {
    locations: ['us-east-1', 'eu-west-1'],
    playwrightConfig: {
      timeout: 30000,
      use: {
        baseURL: 'https://www.checklyhq.com',      
        ...devices['iPhone 12 Pro Max landscape'],
        extraHTTPHeaders: {
          'x-my-custom-header': 'value',        
        }
      }
    }, 
    checkMatch: ['**/__checks__/**/*.check.ts'],
    browserChecks: {
      testMatch: ['**/__checks__/**/*.spec.ts'],
    },   
  },
})

export default config

This is our first step to fully support all global configuration options. We still have some dragons to slay to support globalSetup, globalTeardown, storageState and dependencies but we are working it!

Check our docs for more information 👇

• Using Playwright global configuration with the CLI

• Playwright config options support matrix

Alert Escalation Policies

You can now manage your threshold based alerting with the new alertEscalationPolicy and it's friend the AlertEscalationBuilder.

import { MultiStepCheck, AlertEscalationBuilder } from "checkly/constructs"

const multi = new MultiStepCheck('multi-step-check-1', {
    name: 'Multi step check 1',
    locations: ['eu-west-1'],
    frequency: 10,
    alertEscalationPolicy: AlertEscalationBuilder
        .timeBasedEscalation(5, { amount: 2, interval: 5 }),
    code: {
        entrypoint: 'multi.spec.ts',
    }
})

The new alertEscalationPolicy property is available at the global project level, the CheckGroup and individual Check level. Learn more in the docs.

Other improvements

• The checkMatch and testMatch properties now accept an array of paths so you can more easily organise your files in your repo, i.e.
  

  checkMatch: ['**/checks/**/*.check.ts', '**/more-checks/**/*.check.ts']

• We now also print the test session URL to your terminal when using the dot and ci reporters.

See the all PRs merged on GitHub

To get started with the Checkly CLI just run

npm create checkly 

Or upgrade your existing installation with

npm install checkly@latest

Check the full release note on GitHub here

Visual regression & snapshot testing using Playwright is now GA.

With one line of code, you can now check visual regressions for your sites, apps and component libraries in production and get alerted when things break.

No extra tokens, wrappers or confusing config needed.

import { test, expect } from '@playwright/test';

test('Google search homepage', async ({ page }) => {
   await page.goto('https://google.com')
   await expect(page).toHaveScreenshot()
})

It's super simple:

1. Add an expect().toHaveScreenshot() or expect().toMatchSnapshot() line to any browser check.

2. Create a golden image with one click.

3. Run your check around the clock. Checkly takes care of storing your images in our cloud.

Using our CLI, you can manage this all from your code base. Just add the --update-snapshots flag to your test command.

npx checkly test --update-snapshots

When your snapshot comparison fails, you can inspect it in the handy diff viewer.

To get going with snapshot testing using our CLI and / or the Checkly agent on your private location, make sure to upgrade to the latest version.

• Checkly CLI v4.5.2

• Checkly Agent v3.2.0

Next steps

• Check out the full docs right here

• Check out our video which demos this feature

• Read up on our blog

This feature is available Team and Enterprise plans.

Questions or feedback? Join our Slack community.

All traffic from Checkly's synthetic checks (Browser, API and Multistep) is now coming from a set of static IPs.

This makes it possible to allowlist Checkly traffic in your firewalls and other network appliances and/or tools. You can fetch the range of IPs from a set of handy API endpoints to help with automating this process.

For more info on how to set this up, check out the links below:

• Allowlisting and filtering traffic

• IP information and versions

This feature is now available across the full Checkly network architecture and is available to all users on all plans.

Questions or feedback? Join our Slack community.

Checkly now supports running checks in parallel from multiple locations. Parallel scheduling allows for faster discovery of regional outages and ensures you are always aware of the status of your service from every corner of the world.

Parallel scheduling is an addition to our existing scheduling strategy of round-robin, where each check run is executed on a single location. Compared to round-robin, parallel scheduling can allow you to detect regional outages up to 20x faster than before, depending on how many locations you are monitoring.

Alerts are sent as soon as a location is registered as failing or degrading, minimizing the time from failure to your engineers being notified.

Running a check in parallel

You can select if a check should be executed in parallel from all locations or one at a time, round-robin style, in the check or group settings. Just go to ‘Scheduling & Locations’ when editing a check.

The scheduling strategy can also be configured via the CLI:

import { ApiCheck, AssertionBuilder } from 'checkly/constructs'

new ApiCheck('hello-api-1', {
  name: 'Hello API',
  runParallel: true,
  locations: ['eu-central-1', 'us-east-1'],
  request: {
    method: 'GET',
    url: 'https://mac-demo-repo.vercel.app/api/hello',
    assertions: [
      AssertionBuilder.statusCode().equals(200)
    ],
  }
})

Live update

When scheduling a parallel check, the UI lets you view the progress, as the check run completes per location.

Parallel check runs already include support for the CLI, Terraform provider, public API, Private locations, and more. Note that during beta, dashboards are not fully supported.

Pricing and availability

• Parallel check scheduling is available on all current plans: Hobby, Team, and Enterprise. Deprecated plan types are not supported.

• Each location selected for a check running in parallel does a full check run per execution. An API check that runs in five locations will be billed for five check runs per execution.

Additional resources

• Global locations and scheduling documentation

• CLI documentation

• Terraform provider documentation

• If you have questions or feedback, join our Slack community and let us know.

We wish you happy holidays and look forward to hearing what you think of parallel scheduling!