以前、テスト自動化ツールについて書きましたこちらの続きです。
さらに調べを進めると、PlaywrightはArtilleryという負荷テストと組み合わせて使用できるとのこと。
Artilleryは日本語では「大砲」を意味し、yamlファイルを使用してテストシナリオを作成できる負荷試験ツールです。
Playwrightと組み合わせることでWeb UIをテストすることも可能です。
PlaywrightエンジンはArtilleryに組み込まれており、別途インストールする必要はありません。
早速インストールして、Playwrightのシナリオを使用して起動してみます。
1. Node.js をインストールします。（詳細は割愛）
2. Artillery用のディレクトリでインストールコマンドを実行
$ npm init artillery@latest
3. インストールできたか確認
$ npx artillery dino
------------
< Artillery! >
------------
\
\
__
/ _)
_/\/\/\_/ /
_| /
_| ( | ( |
/__.-'|_|--|_|
恐竜が表示されました！成功です。
4. Playwright のテストシナリオの書き換え
▼Playwrightのテストケース（example.spec.js）
const { test, expect } = require('@playwright/test');
test('has title', async ({ page }) => {
await page.goto('https://playwright.dev/');
// Expect a title "to contain" a substring.
await expect(page).toHaveTitle(/Playwright/);
});
test('get started link', async ({ page }) => {
await page.goto('https://playwright.dev/');
// Click the get started link.
await page.getByRole('link', { name: 'Get started' }).click();
// Expects page to have a heading with the name of Installation.
await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
});
↓
▼ Artillery用に書き換え（example.js）
const { expect } = require("@playwright/test");
☝️requireから test を外す
async function testFlow(page, vuContext, events, test) {
☝️テストケースを関数化する
const { step } = test;
☝️テストケースを細分化するためSTEP化する await step('has title', async() => {
☝️STEPの書き換え 元：test('has title', async ({ page }) => {
await page.goto('/');
☝️テスト対象のURLは yml に記載するのでルート相対に変更
// Expect a title "to contain" a substring.
await expect(page).toHaveTitle(/Playwright/);
});
await step('get started link', async() => {
await page.goto('/');
// Click the get started link.
await page.getByRole('link', { name: 'Get started' }).click();
// Expects page to have a heading with the name of Installation.
await expect(page.getByRole('heading', { name: 'Installation' })).toBeVisible();
});
}
module.exports = { testFlow };
☝️関数を export
5. 負荷テストの条件をYMLに設定
exmple.yml
config:
target: 'https://playwright.dev/'
☝️テスト対象のURLはこちらに記載
engines:
playwright:
launchOptions:
headless: true
trace: false
processor: './example.js'
phases:
# 3秒間に毎秒2ユーザーがアクセスする
- duration: 3
arrivalRate: 2
scenarios:
- engine: playwright
testFunction: 'testFlow'
☝️エンジンとテスト関数を指定
6. 実行
$ npx artillery run ｛ymlファイルのパス｝
7. 結果
Test run id: tdyqz_6ytw4c6xkpcewwzxgzz7nx7kanbjh_jrhd
Phase started: unnamed (index: 0, duration: 3s) 16:11:10(+0900)
Phase completed: unnamed (index: 0, duration: 3s) 16:11:13(+0900)
--------------------------------------
Metrics for period to: 16:11:20(+0900) (width: 4.315s)
--------------------------------------
browser.http_requests: ......................................................... 348
browser.page.CLS.https://playwright.dev/:
min: ......................................................................... 0
max: ......................................................................... 0
mean: ........................................................................ 0
median: ...................................................................... 0
p95: ......................................................................... 0
p99: ......................................................................... 0
browser.page.FCP.https://playwright.dev/:
min: ......................................................................... 117.2
max: ......................................................................... 710.3
mean: ........................................................................ 385.6
median: ...................................................................... 162.4
p95: ......................................................................... 658.6
p99: ......................................................................... 658.6
browser.page.FID.https://playwright.dev/docs/intro:
min: ......................................................................... 10.4
max: ......................................................................... 16.2
mean: ........................................................................ 12.4
median: ...................................................................... 11.8
p95: ......................................................................... 13.1
p99: ......................................................................... 13.1
browser.page.LCP.https://playwright.dev/:
min: ......................................................................... 117.2
max: ......................................................................... 710.3
mean: ........................................................................ 385.6
median: ...................................................................... 162.4
p95: ......................................................................... 658.6
p99: ......................................................................... 658.6
browser.page.TTFB.https://playwright.dev/:
min: ......................................................................... 0.7
max: ......................................................................... 338.1
mean: ........................................................................ 107
median: ...................................................................... 1.3
p95: ......................................................................... 290.1
p99: ......................................................................... 290.1
browser.page.codes.200: ........................................................ 348
browser.step.get started link:
min: ......................................................................... 442
max: ......................................................................... 876
mean: ........................................................................ 609.2
median: ...................................................................... 518.1
p95: ......................................................................... 788.5
p99: ......................................................................... 788.5
browser.step.has title:
min: ......................................................................... 1345
max: ......................................................................... 1728
mean: ........................................................................ 1558.2
median: ...................................................................... 1525.7
p95: ......................................................................... 1686.1
p99: ......................................................................... 1686.1
vusers.completed: .............................................................. 6
vusers.created: ................................................................ 6
vusers.created_by_name.0: ...................................................... 6
vusers.failed: ................................................................. 0
vusers.session_length:
min: ......................................................................... 2131.6
max: ......................................................................... 2592.6
mean: ........................................................................ 2344.2
median: ...................................................................... 2276.1
p95: ......................................................................... 2465.6
p99: ......................................................................... 2465.6
All VUs finished. Total time: 6 seconds
--------------------------------
Summary report @ 16:11:18(+0900)
--------------------------------
browser.http_requests: ......................................................... 348
browser.page.CLS.https://playwright.dev/:
min: ......................................................................... 0
max: ......................................................................... 0
mean: ........................................................................ 0
median: ...................................................................... 0
p95: ......................................................................... 0
p99: ......................................................................... 0
browser.page.FCP.https://playwright.dev/:
min: ......................................................................... 117.2
max: ......................................................................... 710.3
mean: ........................................................................ 385.6
median: ...................................................................... 162.4
p95: ......................................................................... 658.6
p99: ......................................................................... 658.6
browser.page.FID.https://playwright.dev/docs/intro:
min: ......................................................................... 10.4
max: ......................................................................... 16.2
mean: ........................................................................ 12.4
median: ...................................................................... 11.8
p95: ......................................................................... 13.1
p99: ......................................................................... 13.1
browser.page.LCP.https://playwright.dev/:
min: ......................................................................... 117.2
max: ......................................................................... 710.3
mean: ........................................................................ 385.6
median: ...................................................................... 162.4
p95: ......................................................................... 658.6
p99: ......................................................................... 658.6
browser.page.TTFB.https://playwright.dev/:
min: ......................................................................... 0.7
max: ......................................................................... 338.1
mean: ........................................................................ 107
median: ...................................................................... 1.3
p95: ......................................................................... 290.1
p99: ......................................................................... 290.1
browser.page.codes.200: ........................................................ 348
browser.step.get started link:
min: ......................................................................... 442
max: ......................................................................... 876
mean: ........................................................................ 609.2
median: ...................................................................... 518.1
p95: ......................................................................... 788.5
p99: ......................................................................... 788.5
browser.step.has title:
min: ......................................................................... 1345
max: ......................................................................... 1728
mean: ........................................................................ 1558.2
median: ...................................................................... 1525.7
p95: ......................................................................... 1686.1
p99: ......................................................................... 1686.1
vusers.completed: .............................................................. 6
vusers.created: ................................................................ 6
vusers.created_by_name.0: ...................................................... 6
vusers.failed: ................................................................. 0
vusers.session_length:
min: ......................................................................... 2131.6
max: ......................................................................... 2592.6
mean: ........................................................................ 2344.2
median: ...................................................................... 2276.1
p95: ......................................................................... 2465.6
p99: ......................................................................... 2465.6
結果のメトリクスについては、こちらの通りです。
| メトリック | 説明 |
| vusers.created | 作成した仮想ユーザーの数 |
| vusers.completed | 完了した仮想ユーザーの数 |
| vusers.failed | 失敗した仮想ユーザーの数 |
| vusers.session_length | 仮想ユーザーが各セッションを完了するまでの時間（ミリ秒） min（最小値）、max（最大値）、mean（平均値）、median（中央値）、p95（95パーセンタイル）、p99（99パーセンタイル） |
| browser.http_requests | 実行したHTTPリクエストの数 |
| browser.page.FCP | 最初のコンテンツが画面に表示されるまでの時間（ミリ秒） FCP：First Contentful Paint |
| browser.page.FID | クリックなど最初のユーザー操作に対するブラウザの応答時間（ミリ秒） FID：First Input Delay |
| browser.page.LCP | 最も大きな画像またはテキストブロックが表示されるまでの時間（ミリ秒） LCP：Largest Contentful Paint |
| browser.page.TTFB | 最初のレスポンスがブラウザに到着するまでの時間（ミリ秒） TTFB：Time To First Byte |
8. 実行（JSONログ付き）
$ npx artillery run ｛ymlファイルのパス｝--output report.json
9. レポートをHTMLに変換
$ npx artillery report report.json
同時実行数があまりに多いと、テスト実行するPCのスペックに引っ張られて失敗します・・・。
そんな時には、serverless-artilleryを活用することでより高い負荷をかけたテストが可能とのこと。いつかまた挑戦してみたいです。
