test: deterministic operator-guide screenshot harness
integration_test/guide_shots_test.dart boots a hermetic hub
(HubFixture), seeds demo projects (open Bürgeramt; sealed
Ratsinformation and a setup.applied audit event only when the
runner confirms an isolated $HOME), launches the app in German +
dark mode, walks every sidebar page in Cmd order with
content-aware waits, opens the workspace switcher and the setup
wizard, and writes the guide PNGs via a driverless RepaintBoundary
capture. Driven by the platform repo's scripts/regen-studio-guide.sh.
Also fixes the hub fixture's binary resolution, dead since the
rename (it looked for 'fai' and ../chain_platform/): now $CHAIN_BIN,
'chain' on PATH, then ../fai_chain/target/{release,debug}/chain —
the integration tests actually run again instead of silently
skipping.
Signed-off-by: flemming-it <stefan.a.flemming@googlemail.com>
This commit is contained in:
parent
ed2c5ace26
commit
cf4024a4e2
6 changed files with 346 additions and 16 deletions
37
CHANGELOG.md
37
CHANGELOG.md
|
|
@ -5,6 +5,43 @@ version + `kStudioVersion` in `lib/main.dart` stay in lockstep.
|
|||
|
||||
## Unreleased
|
||||
|
||||
### Added (guided setup on grade-1 — steps A4/A5/B1 + doc automation)
|
||||
|
||||
- **Clickable next steps.** After apply, the wizard renders real
|
||||
Studio actions instead of CLI text: start-hub button (polls until
|
||||
the daemon answers), per-module install buttons with progress/done
|
||||
states (capability-name install via the hub's store index), and an
|
||||
open-the-starter-flow button navigating to Flows.
|
||||
- **Signature dead end resolved.** Regulated plans explain in plain
|
||||
language that modules come from a signed source; the preview offers
|
||||
"allow installing from the public store" as one deliberate,
|
||||
reversible switch (`allow_unsigned_modules`) that re-assembles the
|
||||
plan. Air-gapped plans point to the offline-bundle path instead.
|
||||
- **Fresh-install auto-open + honest framing.** On a fresh hub (no
|
||||
config, no `setup-plan.yaml`) the wizard opens by itself, once per
|
||||
run; the welcome CTA is now "In 3 Fragen loslegen" / "Get started
|
||||
in 3 questions". After the wizard closes, the onboarding checklist
|
||||
re-probes and states what the assistant already covered (applied
|
||||
profile from `setup-plan.yaml`).
|
||||
- **Free-text AI path (phase 1.2).** "Or just describe what you want
|
||||
to do": the goal goes to the configured system AI, the reply is
|
||||
validated against strict enum whitelists and comes back as an
|
||||
editable "this is how I read your task" reflection feeding the same
|
||||
preview/apply. Privacy line states local vs. provider processing;
|
||||
without a system AI the menu path stands alone.
|
||||
- **Nav manifest guard.** `test/nav_manifest_test.dart` generates
|
||||
`docs/nav.generated.json` from the sidebar truth (ids, order = Cmd
|
||||
numbers, DE+EN labels); the platform repo checks the operator guide
|
||||
against the mirrored copy.
|
||||
- **Guide screenshot harness.** `integration_test/guide_shots_test.dart`
|
||||
boots a hermetic hub, seeds demo projects, walks every nav page,
|
||||
the workspace switcher and the setup wizard, and writes the guide
|
||||
PNGs — driven by the platform repo's `scripts/regen-studio-guide.sh`.
|
||||
- **Fixed:** the integration-test hub fixture still looked for the
|
||||
pre-rename `fai` binary and `chain_platform/` path, so its tests
|
||||
silently skipped since the rename; it now resolves `$CHAIN_BIN`,
|
||||
`chain` on PATH, and `../fai_chain/target/{release,debug}/chain`.
|
||||
|
||||
### Added (multi-project, stage ③ — sealed areas)
|
||||
|
||||
- **Sealed-area connection switch.** The workspace switcher now lists
|
||||
|
|
|
|||
242
integration_test/guide_shots_test.dart
Normal file
242
integration_test/guide_shots_test.dart
Normal file
|
|
@ -0,0 +1,242 @@
|
|||
// Deterministic operator-guide screenshot harness (one command
|
||||
// instead of the fragile cliclick/screencapture procedure).
|
||||
//
|
||||
// Boots a hermetic hub (HubFixture: temp data dir, free port),
|
||||
// seeds deterministic demo data via the CLI (an open "Bürgeramt"
|
||||
// project; optionally a sealed "Ratsinformation" when the runner
|
||||
// confirms an isolated $HOME), launches the real app in German +
|
||||
// dark mode against that hub, walks every sidebar destination in
|
||||
// nav order, opens the workspace switcher and the guided-setup
|
||||
// wizard, and writes the guide PNGs via a RepaintBoundary capture
|
||||
// (driverless — works headed on macOS with plain `flutter test`).
|
||||
//
|
||||
// Run through the platform repo's scripts/regen-studio-guide.sh,
|
||||
// which isolates $HOME, builds the hub binary, and copies the
|
||||
// PNGs into fai_chain/docs/studio/images/. Direct invocation:
|
||||
//
|
||||
// flutter test integration_test/guide_shots_test.dart -d macos
|
||||
//
|
||||
// Output lands in build/guide-shots/ unless GUIDE_SHOTS_OUT is
|
||||
// set in the environment.
|
||||
//
|
||||
// Hermeticity notes:
|
||||
// - SharedPreferences is mocked empty — the operator's real Studio
|
||||
// prefs are neither read nor written.
|
||||
// - The app reconnects to the fixture hub with persist: false.
|
||||
// - The sealed seed writes to $HOME/.chain/sealed/ by design, so
|
||||
// it only runs when CHAIN_GUIDE_SHOTS_SEALED=1 confirms the
|
||||
// caller isolated $HOME (the regen script does).
|
||||
|
||||
import 'dart:io';
|
||||
import 'dart:ui' as ui;
|
||||
|
||||
import 'package:flutter/material.dart';
|
||||
import 'package:flutter/rendering.dart';
|
||||
import 'package:flutter/services.dart' show LogicalKeyboardKey;
|
||||
import 'package:flutter_test/flutter_test.dart';
|
||||
import 'package:integration_test/integration_test.dart';
|
||||
import 'package:shared_preferences/shared_preferences.dart';
|
||||
|
||||
import 'package:chain_client_sdk/chain_client_sdk.dart' show HubEndpoint;
|
||||
import 'package:chain_studio/data/hub.dart';
|
||||
import 'package:chain_studio/main.dart';
|
||||
import 'package:chain_studio/widgets/widgets.dart';
|
||||
|
||||
import '../test/integration/hub_fixture.dart';
|
||||
|
||||
final GlobalKey _shotKey = GlobalKey();
|
||||
|
||||
/// Pump a fixed number of frames instead of pumpAndSettle: the
|
||||
/// sidebar clock ticks every second, so the tree never settles and
|
||||
/// pumpAndSettle degenerates into its multi-minute timeout.
|
||||
Future<void> _pumpFrames(WidgetTester tester, [int frames = 20]) async {
|
||||
for (var i = 0; i < frames; i++) {
|
||||
await tester.pump(const Duration(milliseconds: 100));
|
||||
}
|
||||
}
|
||||
|
||||
/// Pump until [finder] matches (page content arrived) or ~10 s
|
||||
/// pass — pages load their data from the hub asynchronously, and a
|
||||
/// blind fixed wait raced the slower ones (the flows editor).
|
||||
Future<void> _pumpUntil(
|
||||
WidgetTester tester,
|
||||
Finder finder, {
|
||||
int maxFrames = 100,
|
||||
}) async {
|
||||
for (var i = 0; i < maxFrames; i++) {
|
||||
if (finder.evaluate().isNotEmpty) return;
|
||||
await tester.pump(const Duration(milliseconds: 100));
|
||||
}
|
||||
}
|
||||
|
||||
String get _outDir =>
|
||||
Platform.environment['GUIDE_SHOTS_OUT'] ?? 'build/guide-shots';
|
||||
|
||||
/// Rasterize the app's RepaintBoundary into `<outDir>/<name>.png`.
|
||||
Future<void> _shot(WidgetTester tester, String name) async {
|
||||
// Two extra frames so ripples/route transitions settle visually.
|
||||
await tester.pump(const Duration(milliseconds: 120));
|
||||
await tester.pump(const Duration(milliseconds: 120));
|
||||
final boundary =
|
||||
_shotKey.currentContext!.findRenderObject() as RenderRepaintBoundary;
|
||||
final image = await boundary.toImage(pixelRatio: 2.0);
|
||||
final bytes = await image.toByteData(format: ui.ImageByteFormat.png);
|
||||
image.dispose();
|
||||
final file = File('$_outDir/$name.png');
|
||||
file.parent.createSync(recursive: true);
|
||||
file.writeAsBytesSync(bytes!.buffer.asUint8List());
|
||||
// ignore: avoid_print
|
||||
print('guide-shot: ${file.path}');
|
||||
}
|
||||
|
||||
Future<void> _cli(
|
||||
String binary,
|
||||
HubFixture fixture,
|
||||
List<String> args,
|
||||
) async {
|
||||
final r = await Process.run(
|
||||
binary,
|
||||
args,
|
||||
environment: {
|
||||
...Platform.environment,
|
||||
'CHAIN_DATA_DIR': fixture.tempDir.path,
|
||||
'CHAIN_MODULES_DIR': '${fixture.tempDir.path}/modules',
|
||||
},
|
||||
);
|
||||
if (r.exitCode != 0) {
|
||||
fail('seed command `chain ${args.join(' ')}` failed: ${r.stderr}');
|
||||
}
|
||||
}
|
||||
|
||||
/// The guide's image set, in sidebar (= Cmd number) order. Must
|
||||
/// stay aligned with docs/nav.generated.json — the nav guard in
|
||||
/// the platform repo keeps the guide itself honest.
|
||||
const _pageShots = [
|
||||
('welcome', '01-willkommen'),
|
||||
('store', '02-store'),
|
||||
('doctor', '03-diagnose'),
|
||||
('flows', '04-flows'),
|
||||
('audit', '05-protokoll'),
|
||||
('approvals', '06-freigaben'),
|
||||
('runs', '07-laeufe'),
|
||||
('federation', '08-foederation'),
|
||||
];
|
||||
|
||||
void main() {
|
||||
IntegrationTestWidgetsFlutterBinding.ensureInitialized();
|
||||
|
||||
testWidgets('capture the operator-guide screenshots', (tester) async {
|
||||
final fixture = await HubFixture.start(skipIfBinaryMissing: false);
|
||||
addTearDown(fixture!.dispose);
|
||||
final binary = (await HubFixture.binaryPath())!;
|
||||
|
||||
// Deterministic demo data. The slug is explicit — the derived
|
||||
// one would drop the umlaut ("brgeramt").
|
||||
await _cli(binary, fixture, [
|
||||
'project',
|
||||
'create',
|
||||
'Bürgeramt',
|
||||
'--slug',
|
||||
'buergeramt',
|
||||
'--color',
|
||||
'#2e8f9e',
|
||||
]);
|
||||
// The sealed area writes a manifest under $HOME/.chain/sealed/,
|
||||
// so it needs the runner's confirmation that $HOME is isolated.
|
||||
final sealedSeed =
|
||||
Platform.environment['CHAIN_GUIDE_SHOTS_SEALED'] == '1';
|
||||
if (sealedSeed) {
|
||||
await _cli(binary, fixture, [
|
||||
'project',
|
||||
'create',
|
||||
'Ratsinformation',
|
||||
'--slug',
|
||||
'ratsinformation',
|
||||
'--isolation',
|
||||
'sealed',
|
||||
]);
|
||||
// Seed one audit event the guide can show (setup.applied) by
|
||||
// applying a minimal guided setup against the fixture's data
|
||||
// dir. Writes $HOME/.chain/config.yaml, hence the isolated-
|
||||
// $HOME guard: only the regen script's throwaway HOME is
|
||||
// ever touched.
|
||||
final answers = File('${fixture.tempDir.path}/seed-answers.yaml');
|
||||
answers.writeAsStringSync(
|
||||
'scenario: trying-out\nintent: hello-world\ntarget: this-laptop\n',
|
||||
);
|
||||
await _cli(binary, fixture, [
|
||||
'init',
|
||||
'--answers',
|
||||
answers.path,
|
||||
'--apply',
|
||||
'--force',
|
||||
]);
|
||||
}
|
||||
|
||||
// Hermetic prefs: never read or write the operator's real ones.
|
||||
SharedPreferences.setMockInitialValues({});
|
||||
|
||||
await tester.pumpWidget(
|
||||
RepaintBoundary(
|
||||
key: _shotKey,
|
||||
child: const StudioApp(
|
||||
initialThemeMode: ThemeModeValue.dark,
|
||||
initialLocale: Locale('de'),
|
||||
),
|
||||
),
|
||||
);
|
||||
await HubService.instance.reconnect(
|
||||
HubEndpoint(host: '127.0.0.1', port: fixture.port),
|
||||
authToken: null,
|
||||
persist: false,
|
||||
);
|
||||
await _pumpFrames(tester, 30);
|
||||
|
||||
final shell =
|
||||
tester.state<StudioShellState>(find.byType(StudioShell));
|
||||
|
||||
// 01–08: every sidebar destination, in Cmd order. Where a page
|
||||
// loads slower content from the hub, wait for a sentinel that
|
||||
// proves the content arrived before capturing.
|
||||
for (final (id, shotName) in _pageShots) {
|
||||
shell.navigateTo(id);
|
||||
await _pumpFrames(tester);
|
||||
if (id == 'flows') {
|
||||
// The imported sample flows populate the file list.
|
||||
await _pumpUntil(tester, find.textContaining('extract-summarize'));
|
||||
} else if (id == 'audit') {
|
||||
// The seeded setup.applied event (isolated-$HOME runs).
|
||||
await _pumpUntil(tester, find.textContaining('setup.applied'));
|
||||
}
|
||||
await _shot(tester, shotName);
|
||||
}
|
||||
|
||||
// 09: the workspace switcher, opened (shows the demo projects;
|
||||
// with the sealed seed also the shield entry).
|
||||
shell.navigateTo('audit');
|
||||
await _pumpFrames(tester);
|
||||
final switcher = find.byType(ChainWorkspaceSwitcher);
|
||||
if (switcher.evaluate().isNotEmpty) {
|
||||
await tester.tap(switcher.first);
|
||||
await _pumpFrames(tester);
|
||||
await _shot(tester, '09-projekt-umschalter');
|
||||
// Close the menu again before moving on.
|
||||
await tester.sendKeyEvent(LogicalKeyboardKey.escape);
|
||||
await _pumpFrames(tester, 5);
|
||||
}
|
||||
|
||||
// 11: the guided-setup wizard, step 1 (localized option cards).
|
||||
// 10 (inside a sealed area) still needs the manual procedure —
|
||||
// it requires switching to a second, isolated hub instance.
|
||||
shell.navigateTo('welcome');
|
||||
await _pumpFrames(tester);
|
||||
final cta = find.text('In 3 Fragen loslegen');
|
||||
if (cta.evaluate().isNotEmpty) {
|
||||
await tester.tap(cta.first);
|
||||
await _pumpFrames(tester);
|
||||
await _shot(tester, '11-einrichtung');
|
||||
await tester.sendKeyEvent(LogicalKeyboardKey.escape);
|
||||
await _pumpFrames(tester, 5);
|
||||
}
|
||||
});
|
||||
}
|
||||
|
|
@ -13,4 +13,4 @@ SPEC CHECKSUMS:
|
|||
|
||||
PODFILE CHECKSUM: 54d867c82ac51cbd61b565781b9fada492027009
|
||||
|
||||
COCOAPODS: 1.16.2
|
||||
COCOAPODS: 1.17.0
|
||||
|
|
|
|||
39
pubspec.lock
39
pubspec.lock
|
|
@ -172,6 +172,11 @@ packages:
|
|||
url: "https://pub.dev"
|
||||
source: hosted
|
||||
version: "0.3.5"
|
||||
flutter_driver:
|
||||
dependency: transitive
|
||||
description: flutter
|
||||
source: sdk
|
||||
version: "0.0.0"
|
||||
flutter_highlight:
|
||||
dependency: transitive
|
||||
description:
|
||||
|
|
@ -219,6 +224,11 @@ packages:
|
|||
description: flutter
|
||||
source: sdk
|
||||
version: "0.0.0"
|
||||
fuchsia_remote_debug_protocol:
|
||||
dependency: transitive
|
||||
description: flutter
|
||||
source: sdk
|
||||
version: "0.0.0"
|
||||
google_cloud:
|
||||
dependency: transitive
|
||||
description:
|
||||
|
|
@ -291,6 +301,11 @@ packages:
|
|||
url: "https://pub.dev"
|
||||
source: hosted
|
||||
version: "4.1.2"
|
||||
integration_test:
|
||||
dependency: "direct dev"
|
||||
description: flutter
|
||||
source: sdk
|
||||
version: "0.0.0"
|
||||
intl:
|
||||
dependency: "direct main"
|
||||
description:
|
||||
|
|
@ -435,6 +450,14 @@ packages:
|
|||
url: "https://pub.dev"
|
||||
source: hosted
|
||||
version: "2.1.8"
|
||||
process:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: process
|
||||
sha256: c6248e4526673988586e8c00bb22a49210c258dc91df5227d5da9748ecf79744
|
||||
url: "https://pub.dev"
|
||||
source: hosted
|
||||
version: "5.0.5"
|
||||
protobuf:
|
||||
dependency: transitive
|
||||
description:
|
||||
|
|
@ -544,6 +567,14 @@ packages:
|
|||
url: "https://pub.dev"
|
||||
source: hosted
|
||||
version: "1.4.1"
|
||||
sync_http:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: sync_http
|
||||
sha256: "7f0cd72eca000d2e026bcd6f990b81d0ca06022ef4e32fb257b30d3d1014a961"
|
||||
url: "https://pub.dev"
|
||||
source: hosted
|
||||
version: "0.3.1"
|
||||
term_glyph:
|
||||
dependency: transitive
|
||||
description:
|
||||
|
|
@ -664,6 +695,14 @@ packages:
|
|||
url: "https://pub.dev"
|
||||
source: hosted
|
||||
version: "1.1.1"
|
||||
webdriver:
|
||||
dependency: transitive
|
||||
description:
|
||||
name: webdriver
|
||||
sha256: "2f3a14ca026957870cfd9c635b83507e0e51d8091568e90129fbf805aba7cade"
|
||||
url: "https://pub.dev"
|
||||
source: hosted
|
||||
version: "3.1.0"
|
||||
win32:
|
||||
dependency: transitive
|
||||
description:
|
||||
|
|
|
|||
|
|
@ -50,6 +50,8 @@ dependencies:
|
|||
dev_dependencies:
|
||||
flutter_test:
|
||||
sdk: flutter
|
||||
integration_test:
|
||||
sdk: flutter
|
||||
flutter_lints: ^6.0.0
|
||||
|
||||
# Local-development override: while editing the swappable flow
|
||||
|
|
|
|||
|
|
@ -71,13 +71,13 @@ class HubFixture {
|
|||
if (binary == null) {
|
||||
if (skipIfBinaryMissing) {
|
||||
markTestSkipped(
|
||||
'Hub fixture skipped: no `fai` binary found on PATH or in '
|
||||
'../chain_platform/target/release/fai. Build it with '
|
||||
'`cargo build --release` and re-run.',
|
||||
'Hub fixture skipped: no `chain` binary found via \$CHAIN_BIN, '
|
||||
'PATH, or ../fai_chain/target/{release,debug}/chain. Build it '
|
||||
'with `cargo build` in the platform repo and re-run.',
|
||||
);
|
||||
return null;
|
||||
}
|
||||
throw StateError('fai binary not found');
|
||||
throw StateError('chain binary not found');
|
||||
}
|
||||
|
||||
final tempDir = await Directory.systemTemp.createTemp('chain_studio_test_');
|
||||
|
|
@ -157,27 +157,37 @@ class HubFixture {
|
|||
}
|
||||
}
|
||||
|
||||
/// Look for `fai` first on PATH (production-ish), then in
|
||||
/// the standard `target/release/` build output relative to
|
||||
/// the platform checkout (developer flow). Returns null when
|
||||
/// neither path resolves.
|
||||
/// Resolved hub binary path, for tests that also drive the CLI
|
||||
/// against the fixture's data dir (e.g. seeding projects).
|
||||
static Future<String?> binaryPath() => _resolveBinary();
|
||||
|
||||
/// Look for the hub binary: $CHAIN_BIN override first, then
|
||||
/// `chain` on PATH (production-ish), then the release/debug
|
||||
/// build outputs relative to the platform checkout (developer
|
||||
/// flow, repo dir `fai_chain`). Returns null when nothing
|
||||
/// resolves.
|
||||
static Future<String?> _resolveBinary() async {
|
||||
final fromEnv = Platform.environment['CHAIN_BIN'];
|
||||
if (fromEnv != null && fromEnv.isNotEmpty && File(fromEnv).existsSync()) {
|
||||
return fromEnv;
|
||||
}
|
||||
|
||||
final pathResult = await Process.run(
|
||||
Platform.isWindows ? 'where' : 'which',
|
||||
['fai'],
|
||||
[Platform.isWindows ? 'chain.exe' : 'chain'],
|
||||
);
|
||||
if (pathResult.exitCode == 0) {
|
||||
final s = (pathResult.stdout as String).trim();
|
||||
if (s.isNotEmpty) return s.split('\n').first.trim();
|
||||
}
|
||||
|
||||
final candidate = Platform.isWindows
|
||||
? '../chain_platform/target/release/fai.exe'
|
||||
: '../chain_platform/target/release/fai';
|
||||
final f = File(candidate);
|
||||
final exe = Platform.isWindows ? 'chain.exe' : 'chain';
|
||||
for (final dir in ['release', 'debug']) {
|
||||
final f = File('../fai_chain/target/$dir/$exe');
|
||||
if (await f.exists()) {
|
||||
return f.absolute.path;
|
||||
}
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue