Changelog

Follow new updates and improvements to Checkly.

February 14th 2024

Fixed

Improved

This patch release of the Checkly CLI fixes some small bugs and adds

Alerting additions

You can now set the alerting threshold for failing locations when using parallel scheduling. in the Web UI this configured like below πŸ‘‡

For the CLI, you add the parallelRunFailureTreshold object to your alertEscalationPolicy:

alertEscalationPolicy: AlertEscalationBuilder
    .runBasedEscalation(2,
      { amount: 2, interval: 5 }, // sets reminders
      { enabled: true, percentage: 50 } // sets the parallelRunFailureThreshold
),

Global configuration fix

We missed exporting the userAgent as an option for global configuration. This option is added now. This means two things:

1. You can set a user agent in your global config, e.g.

use: {
  userAgent: 'my funky user'
}


2. Using the devices macro now sets the correct user agent header. That field was ignored earlier, e.g.

import { devices } from '@playwright/test'

use: {
   ...devices['iPhone 12 Pro Max landscape']
}


the above code will set the user agent header to:

"Mozilla/5.0 (iPhone; CPU iPhone OS 14_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/17.4 Mobile/15E148 Safari/604.1"

Learn more about the Playwright Global Configuration Beta.

To get started with the Checkly CLI just run

npm create checkly 

Or upgrade your existing installation with

npm install checkly@latest


Check the full release note on GitHub here

Type @ to mention posts

January 31st 2024

New

Parallel scheduling is now GA.

Reduce mean time to detection, get better insights when addressing outages, and get improved data granularity on all performance trends with parallel scheduling.

A check using parallel scheduling will run from each configured location each time it is scheduled, ensuring that any local outages are detected instantly. Alerts are sent as soon as a location is registered as failing or degrading, minimizing the time from failure to your team being notified.

Changing your scheduling method

With GA, parallel scheduling is now the default for all new checks and groups. To change the scheduling method, go to "Scheduling & Locations" when editing your check.

Parallel scheduling is fully supported in our monitoring as code workflow. With our CLI, just add the runParallel property to your construct:

import { ApiCheck, AssertionBuilder } from 'checkly/constructs'

new ApiCheck('hello-api-1', {
  name: 'Hello API',
  runParallel: true,
  locations: ['eu-central-1', 'us-east-1'],
  request: {
    method: 'GET',
    url: 'https://mac-demo-repo.vercel.app/api/hello',
    assertions: [
      AssertionBuilder.statusCode().equals(200)
    ],
  }
})


Similar, with Terraform just add run_parallel

resource "checkly_check" "list-all-checks" {
  name                      = "List all checks"
  type                      = "API"
  frequency                 = 0
  frequency_offset          = 10
  activated                 = false
  muted                     = false
  run_parallel              = true
  locations                 = ["eu-north-1", "eu-central-1", "us-west-1", "ap-northeast-1"]
  degraded_response_time    = 5000
  max_response_time         = 20000
  request {
    method                    = "GET"
    url                       = "https://developers.checklyhq.com/reference/getv1checks"
  }
}

Viewing check results

The check overview page will now show the progress of all locations as a check is running, giving you instant feedback if a certain location is taking longer than usual.

The check result page has been updated to support results from multiple locations. You can switch between locations in the sidebar when viewing the check result page.

Quickly compare the results from multiple locations, reviewing failures, Playwright traces or screenshots to understand how your service behaves globally.

Pricing and availability

  • Parallel check scheduling is available on all current plans: Hobby, Team, and Enterprise. Deprecated plan types are not supported.

  • Each location selected for a check running in parallel does a full check run per execution. An API check that runs in five locations will be billed for five check runs per execution. For more information on pricing, see our documentation: Check pricing.

Additional resources

Type @ to mention posts

January 26th 2024

New

We now support a subset of global configuration options for our Playwright-powered Browser and Multistep checks via our CLI.

To get started, upgrade your CLI to version 4.6.0 and add any supported options to the new playwrightConfig section in your checkly.config.ts file. To quickly copy the supported settings over to your checkly.config.ts file, you can run...

npx checkly sync-playwright


You should now have a checkly.config.ts file similar to the example below πŸ‘‡

import { defineConfig } from 'checkly'
import { devices } from '@playwright/test'

const config = defineConfig({
  projectName: 'Checkly website',
  logicalId: 'checkly-website-project',
  checks: {
    locations: ['us-east-1', 'eu-west-1'],
    playwrightConfig: {
      timeout: 30000,
      use: {
        baseURL: 'https://www.checklyhq.com',      
        ...devices['iPhone 12 Pro Max landscape'],
        extraHTTPHeaders: {
          'x-my-custom-header': 'value',        
        }
      }
    }, 
    checkMatch: ['**/__checks__/**/*.check.ts'],
    browserChecks: {
      testMatch: ['**/__checks__/**/*.spec.ts'],
    },   
  },
})

export default config

Note that we now set the timeout, baseUrl, device properties and extraHTTPHeaders globally for all Browser and Multistep checks using the canonical Playwright configuration options. You can now write a Browser check like...

import { test, expect } from '@playwright/test'

test('Checkly pricing page', async ({ page, baseURL }) => {
  const response = await page.goto(baseURL! + '/pricing')
  expect(response?.status()).toBeLessThan(400)
})


After deploying your checks with npx checkly deploy, you can access the fully rendered configuration in the new "playwright test config"-section in the web UI editor.

This is our first step to fully support all global configuration options. We still have some dragons to slay to support globalSetup, globalTeardown, storageState and dependencies but we are working it!

Check our docs for more information.

Type @ to mention posts

January 26th 2024

New

The latest release of the Checkly CLI brings the following new capabilities and improvements.

Global playwright.config.ts support in Beta

We now support a subset of global configuration options for our Playwright-powered browser and multistep checks. We also have a dedicated changelog entry with all the details.

To get started, just add any supported options to the new playwrightConfig section in your checkly.config.ts file. To quickly copy the supported settings over to your checkly.config.ts file, you can run:

npx checkly sync-playwright


You should then have a config file similar to the example below πŸ‘‡

import { defineConfig } from 'checkly'
import { devices } from '@playwright/test'

const config = defineConfig({
  projectName: 'Checkly website',
  logicalId: 'checkly-website-project',
  checks: {
    locations: ['us-east-1', 'eu-west-1'],
    playwrightConfig: {
      timeout: 30000,
      use: {
        baseURL: 'https://www.checklyhq.com',      
        ...devices['iPhone 12 Pro Max landscape'],
        extraHTTPHeaders: {
          'x-my-custom-header': 'value',        
        }
      }
    }, 
    checkMatch: ['**/__checks__/**/*.check.ts'],
    browserChecks: {
      testMatch: ['**/__checks__/**/*.spec.ts'],
    },   
  },
})

export default config

This is our first step to fully support all global configuration options. We still have some dragons to slay to support globalSetup, globalTeardown, storageState and dependencies but we are working it!

Check our docs for more information πŸ‘‡

Alert Escalation Policies

You can now manage your threshold based alerting with the new alertEscalationPolicy and it's friend the AlertEscalationBuilder.

import { MultiStepCheck, AlertEscalationBuilder } from "checkly/constructs"

const multi = new MultiStepCheck('multi-step-check-1', {
    name: 'Multi step check 1',
    locations: ['eu-west-1'],
    frequency: 10,
    alertEscalationPolicy: AlertEscalationBuilder
        .timeBasedEscalation(5, { amount: 2, interval: 5 }),
    code: {
        entrypoint: 'multi.spec.ts',
    }
})

The new alertEscalationPolicy property is available at the global project level, the CheckGroup and individual Check level. Learn more in the docs.

Other improvements

  • The checkMatch and testMatch properties now accept an array of paths so you can more easily organise your files in your repo, i.e.

    checkMatch: ['**/checks/**/*.check.ts', '**/more-checks/**/*.check.ts']
  • We now also print the test session URL to your terminal when using the dot and ci reporters.

See the all PRs merged on GitHub

To get started with the Checkly CLI just run

npm create checkly 

Or upgrade your existing installation with

npm install checkly@latest


Check the full release note on GitHub here

Type @ to mention posts

January 17th 2024

New

Visual regression & snapshot testing using Playwright is now GA.

With one line of code, you can now check visual regressions for your sites, apps and component libraries in production and get alerted when things break.

No extra tokens, wrappers or confusing config needed.

import { test, expect } from '@playwright/test';

test('Google search homepage', async ({ page }) => {
   await page.goto('https://google.com')
   await expect(page).toHaveScreenshot()
})

It's super simple:

  1. Add an expect().toHaveScreenshot() or expect().toMatchSnapshot() line to any browser check.

  2. Create a golden image with one click.

  3. Run your check around the clock. Checkly takes care of storing your images in our cloud.

Using our CLI, you can manage this all from your code base. Just add the --update-snapshots flag to your test command.

npx checkly test --update-snapshots

When your snapshot comparison fails, you can inspect it in the handy diff viewer.

To get going with snapshot testing using our CLI and / or the Checkly agent on your private location, make sure to upgrade to the latest version.

Next steps


This feature is available Team and Enterprise plans.

Questions or feedback? Join our Slack community.

Type @ to mention posts

January 12th 2024

New

All traffic from Checkly's synthetic checks (Browser, API and Multistep) is now coming from a set of static IPs.

This makes it possible to allowlist Checkly traffic in your firewalls and other network appliances and/or tools. You can fetch the range of IPs from a set of handy API endpoints to help with automating this process.

For more info on how to set this up, check out the links below:

This feature is now available across the full Checkly network architecture and is available to all users on all plans.

Questions or feedback? Join our Slack community.

Type @ to mention posts

December 15th 2023

New

Checkly now supports running checks in parallel from multiple locations. Parallel scheduling allows for faster discovery of regional outages and ensures you are always aware of the status of your service from every corner of the world.

Parallel scheduling is an addition to our existing scheduling strategy of round-robin, where each check run is executed on a single location. Compared to round-robin, parallel scheduling can allow you to detect regional outages up to 20x faster than before, depending on how many locations you are monitoring.

Alerts are sent as soon as a location is registered as failing or degrading, minimizing the time from failure to your engineers being notified.

Running a check in parallel

You can select if a check should be executed in parallel from all locations or one at a time, round-robin style, in the check or group settings. Just go to β€˜Scheduling & Locations’ when editing a check.

The scheduling strategy can also be configured via the CLI:

import { ApiCheck, AssertionBuilder } from 'checkly/constructs'

new ApiCheck('hello-api-1', {
  name: 'Hello API',
  runParallel: true,
  locations: ['eu-central-1', 'us-east-1'],
  request: {
    method: 'GET',
    url: 'https://mac-demo-repo.vercel.app/api/hello',
    assertions: [
      AssertionBuilder.statusCode().equals(200)
    ],
  }
})

Live update

When scheduling a parallel check, the UI lets you view the progress, as the check run completes per location.

Parallel check runs already include support for the CLI, Terraform provider, public API, Private locations, and more. Note that during beta, dashboards are not fully supported.

Pricing and availability

  • Parallel check scheduling is available on all current plans: Hobby, Team, and Enterprise. Deprecated plan types are not supported.

  • Each location selected for a check running in parallel does a full check run per execution. An API check that runs in five locations will be billed for five check runs per execution.

Additional resources

We wish you happy holidays and look forward to hearing what you think of parallel scheduling!

Type @ to mention posts

November 10th 2023

New

Checkly now supports Playwright Test's visual comparison & snapshot testing in our Playwright powered browser checks!

With one line of code, you can now check visual regressions for your sites, apps and component libraries. No extra tokens, wrappers or confusing config needed.

import { test, expect } from '@playwright/test';

test('Google search homepage', async ({ page }) => {
   await page.goto('https://google.com')
   await expect(page).toHaveScreenshot()
})

It's super simple:

  1. Add an expect().toHaveScreenshot() or expect().toMatchSnapshot() line to any browser check.

  2. Create a golden image with one click.

  3. Run your check around the clock. Checkly takes care of storing your images in our cloud.

Using our CLI, you can manage this all from your code base. Just add the --update-snapshots flag to your test command.

npx checkly test --update-snapshots

When your snapshot comparison fails, you can inspect it in the handy diff viewer.

To get going with snapshot testing using our CLI and / or the Checkly agent on your private location, make sure to upgrade to the latest version.

Check out the full docs right here.

This feature is available on all plans during beta. Any pricing or plans limits will be introduced after the beta.

Questions or feedback? Join our Slack community.

Type @ to mention posts

November 7th 2023

New

We are very excited to announce the beta release of Multistep API checks!

Now, you can test and monitor entire user flows by chaining together multiple API requests or test all methods on an endpoint in a single check, all using our web script editor or your IDE of choice.

Why Multistep API checks?

As a fully programmable check type, multistep API checks allow you to mimic real user journeys and actions closely. Change, delete, or add new data between request steps as needed. Unlike traditional multistep solutions, you are not bound to what some form builder allows; it is all done in code you control.

Multistep API checks support the entire Checkly monitoring-as-code workflow from day one; you can integrate existing Playwright API checks today and use our CLI or Terraform provider to run them on Checkly.

Multistep API checks and Browser Checks

While similar to Checkly’s browser checks, multistep API checks provide deep insights into each API request made with a purpose-built result viewer. Multistep checks are also much more affordable, with each request priced at the price of an API check. This allows you to monitor complex user journeys reliably and economically.

Getting started

Similar to browser checks, we use Playwright to power multistep API checks. When creating your first multistep API check, you can choose from one of the existing templates to get started, and if you have existing Playwright API tests, you can now use those to monitor your product.

To provide clear and actionable information, multistep checks rely on the test.step method. When writing a multistep check, we recommend having a test step with a descriptive label for each request required and any assertions related to the request inside the same test step. Here is a simple example.

import { test, expect } from '@playwright/test'

const baseUrl = 'https://api.myproduct.com/v1'

test('My test', async ({request}) => {
    await test.step('First step', async () => {
        const health = await request.get(`${baseUrl}/health`)
        expect(health).toBeOK()
        await expect(response).toBeOK()
    });

    await test.step('Second step', async () => {
        const response = await request.post(`${baseUrl}/health`)
        // Assertions for the second step
        ...
    })
})

When building a new multistep check using our web editor, you can use the test result information in the side panel to view error information, request results, and assertion details. Simply click on any request node to get the full result.

The request result and assertion details are also available in the check result view:

Multistep checks support all Checkly alerting channels, including custom webhooks. The new check type also supports test sessions, dashboards, maintenance windows, and more.

Using the Checkly CLI or Terraform

This first release of multistep API checks includes full support for the Checkly CLI and our Terraform provider, allowing you to include multistep checks in your MaC workflow from day one. The MultistepCheck construct is available in Checkly CLI v4.3.0

import { MultiStepCheck } from 'checkly/constructs'

new MultiStepCheck('multistep-check-1', {
  name: 'Multistep Check #1',
  runtimeId: '2023.09',
  frequency: Frequency.EVERY_10M,
  locations: ['us-east-1', 'eu-west-1'],
  code: {
    entrypoint: path.join(__dirname, 'home.spec.ts')
  },
})

Additional resources

We look forward to hearing what you think of the latest addition to Checkly!

Multistep API checks are available on all Checkly plans. Each request in a multistep API check counts as a single API check for billing purposes.

Type @ to mention posts

October 10th 2023

New

If you are using one of our SSO integrations, it might be hard or even impossible to create "service users" to call the Checkly public API or use our CLI from a background process like CI. This is because API keys are tied to users.

To make life easier here, we just introduced Service API keys. Service API keys work very similar to user API keys, but are different on some parts.

  • Service API keys can have access roles: READ_ONLY, READ_WRITE and ADMIN.

  • Service API keys are tied to one account only. They are not usable for cross account access.

  • Service API keys can only be used for public API access and CLI usage, just like user API keys. Just set the API in your terminal or script, or export it as CHECKLY_API_KEY=sv_...

curl -H "X-Checkly-Account: 438481ea-0eab-43d6-8932-redacted" -H "Authorization: Bearer sv_9afd2cc24a46432bba922-redacted" https://api.checklyhq.com/v1/checks

Check out the full docs right here.

This feature is available on Enterprise plans only.

Questions or feedback? Join our Slack community.

Type @ to mention posts