Beaker

What’s New in Beaker 19?

Beaker 19 brings many improvements to the system page. For Beaker administrators, the upgrade process has been streamlined with automatic database schema upgrades.

Starting with this release, the leading zero has been dropped from Beaker’s version numbering scheme.

Improved system page

In this release the system page has been re-arranged and improved. The aim of the improvements is to reduce wasted space, convey information more efficiently, and simplify interactions with the page.

Important changes you need to be aware of are listed below. For a complete description of all the changes and their background and rationale, refer to the original design proposal.

The new layout relies on browser support for the CSS3 Flexible Box Layout Module. Beaker is tested with Firefox 24 (which is the oldest supported ESR version of Firefox at the time of writing). Older versions of Firefox are not supported and may be unable to render the system page properly.

Major page layout changes

The system form — the set of fields arranged in two columns at the top of the system page — has historically formed the focus of the system page. Over many years of development, however, it has grown into a disorganized assortment of data, of which only a small amount is relevant for any given workflow on the page.

In its place, the system page now has three “quick info boxes”. They are designed to show the most important facts about the system and to give quick access to the most common operations, while occupying a very small amount of vertical space. The left-hand box shows a summary of the system’s hardware. The middle box shows a summary of the system’s current usage. The right-hand box shows a summary of the system’s health.

The interface elements previously contained in the system form will instead be shown in tabs below. The previous horizontal tab strip is replaced with a vertical list of tabs, to accommodate their increasing number. The tabs have also been re-ordered and grouped by theme.

Relocated interface elements

The various fields and interface elements which previously made up the system form are now grouped into more appropriate tabs on the page:

  • The Lab Controller, Location, Lender, and Kernel Type fields are part of the Hardware Essentials tab.
  • The Hypervisor field (renamed Host Hypervisor for clarity), as well as the Vendor, Model, Serial Number, and MAC Address fields, are included in the Hardware Details tab.
  • The Owner and Notify CC fields are located on a new Owner tab.
  • The Loan Settings modal, plus the Request Loan functionality previously accessible through the Contact Owner button, have been moved to a dedicated Loan tab.
  • The Condition and Type fields are part of the Scheduler Settings tab.
  • Change a system’s FQDN by clicking Rename in the page header.

The Arch(s) tab, for specifying supported architectures for the system, has been replaced by the Supported Architectures field on the Hardware Essentials tab.

Provision tab always provisions

The Provision tab now always provisions the system immediately (if you have permission to do so). In previous versions of Beaker, the tab would sometimes schedule a new job for the system instead of provisioning it immediately, depending on the current state of the system.

To provision a system through the scheduler, use the reserve workflow. The Provision tab now includes a direct link to the reserve workflow for the specific system.

Screen scraping scripts will be impacted

The HTML structure of the system page has changed substantially in this release. In addition, a number of widgets render their markup entirely in the browser and no corresponding HTML appears in the server response. Therefore any screen scraping scripts which interact with the system page are likely to be impacted.

Since Beaker 0.15 a number of new Beaker client subcommands for manipulating systems have been added, to reduce the need for screen scraping scripts. You should use these in preference to screen scraping whenever possible:

If you have screen scraping scripts whose functionality is not covered by these subcommands, please file an RFE against Beaker requesting a new client command exposing the functionality you need.

Manual systems in the Reserve Workflow

When the user is browsing systems in the Reserve Workflow, Beaker now also offers systems which are in Manual mode (in addition to Automated). If the user picks a Manual system it will be reserved using the “forced system scheduling” mechanism introduced in Beaker 0.17.

System access policy restrictions will still apply as normal. Users will only be offered systems for which they have reserve permission.

If the user does not pick a specific system, the usual scheduler behaviour will continue to apply: only Automated systems will be selected by the scheduler.

(Contributed by Amit Saha in #%s1093226.)

Automatic database schema upgrades

The beaker-init command now supports fully automatic database schema upgrades and downgrades using Alembic. It can upgrade Beaker databases from version 0.11 or higher.

(Contributed by Matt Jia and Dan Callaghan in #%s682030.)

Notable changes

systemd-readahead is disabled in Beaker recipes

Beaker now disables readahead collection on distros with systemd, in the same way that the readahead service is disabled on RHEL6. Readahead is not generally useful in Beaker recipes because they typically only boot once, and the harness interferes with normal data collection.

You can opt out of this behaviour by setting the no_disable_readahead kickstart metadata variable. This will cause Beaker to omit the snippet which disables readahead collection.

Workflow commands no longer use STABLE

The bkr workflow commands no longer filter for distros tagged STABLE by default. If your Beaker installation is using the STABLE tag, you can apply the filter explicitly by adding --tag=STABLE when invoking workflow commands.

Other new features and enhancements

A new subcommand bkr system-modify has been added to the Beaker client to modify attributes of existing systems. In this initial release, the subcommand can change the system owner and condition. (Contributed by Amit Saha and Dan Callaghan in #%s1118884, #%s804479.)

A new system permission view_power allows users to view and export the power settings for a system. System owners can grant this permission to trusted users/groups for debugging purposes. (Contributed by Dan Callaghan in #%s1012174.)

Task and harness updates

Version 4.0-86 of the /distribution/virt/install task for installing guest recipes has been published. The task no longer disables NetworkManager in favour of the network initscript on distros where NetworkManager is capable of handling bridging (RHEL7 and Fedora). (Contributed by Matt Jia in #%s1150132.)

A new task /distribution/virt/image-install has been published, providing experimental support for running guest recipes in VMs booted from disk images with cloud-init. Refer to the task documentation for more details. (Contributed by Matt Jia in #%s1108455.)

Version 0.7.8 of the Beah test harness has been released, fixing an issue with the harness service configurations for systemd which could cause systemd to enter an inconsistent state. (Contributed by Dan Callaghan in #%s1147807.)

Version 1.5 of the beaker-system-scan utility has been released, fixing a regression which affects systems whose /boot volume is on a mapped block device. (Contributed by Amit Saha in #%s1148174.)

Bug fixes

The following user interface bugs/RFEs are solved by the system page improvements in this release:

  • #%s619335: The Provision tab should offer a way of filtering distros, to make it easier to find the desired distro.
  • #%s692777: The system page should show how long a system has been reserved.
  • #%s880724: The reserve workflow does not filter systems by lab controller, even if you select a specific lab controller when filtering for distro trees.
  • #%s884399: When using the Provision tab, any install options given are applied on top of the default install options for that system and distro. As a consequence, if you edit the pre-populated install options on the Provision tab to remove a default option, it will have no effect.
  • #%s980352: No error message is shown if a validation error occurs when editing a system (for example, when the condition report value is too long).
  • #%s999444: The Loan Settings button appears when editing a system, but clicking it does nothing.
  • #%s1009323: If a user has no permission to edit a system, clicking the Edit system button or the Change button for notify cc redirects the user back to the system list, instead of to the original system page.
  • #%s1011284: The Loan Settings button disappears after returning an existing loan.
  • #%s1011293: The loan settings modal offers to return a loan even when none exists.
  • #%s1020107: After changing loan settings and closing the loan settings modal, the system page does not reflect the new state of the system. In particular, if a user loans the system to themselves they should then be permitted to take the system, but the Take button does not appear.
  • #%s1037280: The meaning of the Hypervisor field on the system page is not clear.
  • #%s1059535: When saving changes on the Power Config tab, all fields are recorded in the system activity as being changed, even if they were not actually changed.
  • #%s1062086: When using the reserve workflow, if the user selects a combination of options which cannot be satisfied by any systems, Beaker warns about the situation but then schedules the job anyway.
  • #%s1062706: The procedure for “taking” an Automated system is awkward and requires too many steps.
  • #%s1070036: When saving changes on the Power Config tab, if a validation error occurs all fields are cleared and the values are lost.
  • #%s1134689: Under some circumstances when saving changes on the Access Policy tab, a rule is recorded as removed and added multiple times for no reason.

A number of other bug fixes are also included in this release:

  • #%s1160513: Fixed a JavaScript TypeError which would occur when viewing the system page for a system whose access policy does not contain any rules. (Contributed by Dan Callaghan)
  • #%s891827: The bkr workflow commands no longer use the STABLE distro tag by default, since it does not exist in default Beaker installations. (Contributed by Dan Callaghan)
  • #%s1142714: The bkr job-submit command now reads job XML from stdin when no positional arguments are given. (Contributed by Dan Callaghan)
  • #%s1032881: The beaker-import command now has a man page. (Contributed by Amit Saha)
  • #%s1142532: Server-side memory usage has been reduced in code paths which create activity records. In particular, this fixes a MemoryError which can occur on large lab controllers when beaker-provision is restarted after being killed uncleanly. (Contributed by Dan Callaghan)
  • #%s802641: Deleting of lab controllers is now much faster and more memory efficient. Previously, attempting to delete a lab controller would time out in very large labs. (Contributed by Dan Callaghan)
  • #%s1069438, #%s1061955: Fixed a number of minor database schema inconsistencies between freshly created databases and existing upgraded databases, caused by mistakes in old release notes. (Contributed by Dan Callaghan)
  • #%s1160091: Kickstart templates have been tweaked to avoid a bash syntax error in case the administrator has defined a custom readahead_sysconfig or virt_console_post snippet with no content. (Contributed by Dan Callaghan)
  • #%s1129059: The notification e-mail sent when a system is reserved with <reservesys/> now includes a link to the recipe, and includes more useful information when the recipe is running on OpenStack. (Contributed by Dan Callaghan)

Maintenance updates

The following fixes have been included in Beaker 19 maintenance updates.

Beaker 19.1

  • #%s1162513: Beaker now adds a PPC PReP Boot partition for recipes using custom partitioning on ppc64le distros. Previously it was only defined for ppc and ppc64 distros, leading to installation failures on ppc64le distros. (Contributed by Dan Callaghan)
  • #%s1172450: Beaker now adds the --leavebootorder option to the bootloader kickstart command on ppc64le distros. Previously it was only added for ppc and ppc64 distros, which caused the boot order to left in an incorrect state after provisioning a ppc64le distro. (Contributed by Amit Saha)
  • #%s1014695: The bkr workflow commands now accept --job-owner. Submission delegates can use this option to submit jobs on behalf of other users. (Contributed by Dan Callaghan)
  • #%s1118523: The bkr list-systems <bkr system-list> command now accepts --host-filter. This option has the same functionality as the --host-filter option for bkr workflow commands. (Contributed by Dan Callaghan)
  • #%s1131429: By default beaker-wizard now suggests excluding RHEL4 and RHEL5 on newly created tasks. (Contributed by Dan Callaghan)
  • #%s902299: beaker-wizard now strips the word “EMBARGOED” from bug summaries when suggesting the task name. (Contributed by Martin Kyral)
  • #%s1165754: beaker-wizard now correctly escapes backticks in shell commands appearing in the generated Makefile, so that they are not evaluated by the shell when make is run. (Contributed by Dan Callaghan)
  • #%s854229: The swapsize kickstart metadata variable now correctly sets the size of the swap partition. Previously it was undocumented and did not work in all cases. (Contributed by Dan Callaghan)
  • #%s745560: The beaker-init utility is now capable of converting an existing user account into an admin. (Contributed by Dan Callaghan)
  • #%s1102617: The special admin group is no longer allowed to be deleted. (Contributed by Matt Jia)
  • #%s949855: Beaker now correctly handles XML host filters which use requirements of the form <device op="!=" driver="drivername"/>. (Contributed by Dan Callaghan)
  • #%s1162451: When a recipe is scheduled with <hostRequires force=""/> on a Manual system, and the system is already reserved, the recipe will remain queued until the system becomes free. Previously Beaker would erroneously consider the queued recipe to be “dead” and abort it. (Contributed by Dan Callaghan)
  • #%s1163721: Fixed a display issue with the system page when the browser’s minimum font size is larger than 14px. (Contributed by Dan Callaghan)
  • #%s1161373: Fixed a number of issues which would cause excessive error messages to appear on the system page under some circumstances. (Contributed by Dan Callaghan)
  • #%s1140912: Beaker now returns a more descriptive error message when an invalid username is given in the user="" attribute of the <job/> element when submitting a job. (Contributed by Dan Callaghan)
  • #%s1167164: When syntactically invalid XML is passed to bkr list-systems --xml-filter, the command now prints a concise error message instead of an HTML error page. (Contributed by Dan Callaghan)
  • #%s1073266: Improved the wording of error messages when the lab controller daemons fail to start. (Contributed by Dan Callaghan)
  • #%s1009377: The --help output for bkr policy-grant and bkr policy-revoke now lists all possible permission values. (Contributed by Dan Callaghan)
  • #%s1142591: The beaker-lab-controller package now correctly depends on syslinux. This allows Beaker to automatically copy pxelinux.0 into the TFTP root directory for convenience in new Beaker installations. (Contributed by Dan Callaghan)

Beaker 19.2

  • #%s1163466: The beaker-provision daemon now generates netboot configuration files for Petitboot, the boot loader used in some recent IBM Power systems. (Contributed by Amit Saha)
  • #%s1169293: Usage reminder e-mails now include a header X-Beaker-Notification: usage-report to aid in mail filtering. (Contributed by Matt Jia)
  • #%s1175118: The beaker-import command now correctly imports the “Everything” repo for Fedora 21. This fixes an issue which would cause Fedora 21 jobs to fail with unsatisfied dependencies. The distro must be re-imported for the fix to take effect. (Contributed by Amit Saha)
  • #%s1174279: Reverted a change in Beaker 19.0 which caused NTP services not to be enabled for guest recipes. Clock syncing is now enabled in all recipes by default. Additionally, guest recipes are now configured so that their hardware clock is assumed to be in UTC, as provided by the host. (Contributed by Matt Jia)
  • #%s1157348: Beaker no longer requires that a cached harness repo exists for the recipe when using the experimental “contained harness” feature. In that case the job submitter is responsible for ensuring suitable harness packages are available. (Contributed by Amit Saha)
  • #%s1173402: Removed references to running Beah inside a container, as part of the experimental “contained harness” feature. This never worked properly due to limitations of Beah. Restraint is the recommended harness inside containers. (Contributed by Amit Saha)
  • #%s1174786: Fixed an issue with the kickstart snippet for handling grubport= which would cause the GRUB menu.lst symlink to be overwritten. (Contributed by Dan Callaghan)
  • #%s1173362: Beaker now prevents the admin from setting an OS major’s alias to the same name or alias as another OS major. (Contributed by Dan Callaghan)
  • #%s1173368: Beaker no longer accepts the empty string as an OS major name or distro name. (Contributed by Matt Jia)
  • #%s963042: The lab controller daemons now properly enforce a 120-second timeout for connections to the Beaker server. Previously the connect timeout did not take effect for https:// connections. (Contributed by Matt Jia)
  • #%s1173446: The Provision tab on the system page now always shows a link to the Reserve Workflow. (Contributed by Dan Callaghan)

Beaker 19.3

  • #%s856687: Beaker client workflow commands accept a new option --ks-append, for appending additional commands to the kickstart. (Contributed by Matt Jia)
  • #%s1124140: The documentation now covers the JSON APIs for systems added since Beaker 0.15. (Contributed by Dan Callaghan)
  • #%s1183239: Fixed incorrect query generation when a recipe’s host requirements contain an <arch/> filter inside an <or/> element. Previously these filters would trigger a runaway database query which could consume all temp table space. (Contributed by Dan Callaghan)
  • #%s1182545: Fixed handling of date formats when searching systems by date added or date inventoried. (Contributed by Matt Jia)
  • #%s1171936: Beaker now detects and reports invalid sh-style syntax in kickstart metadata: for example, an unterminated ' quote. (Contributed by Dan Callaghan)
  • #%s1193746: Fixed an issue with redirects in JSON collection APIs, where query string parameters would be discarded. (Contributed by Dan Callaghan)

Version 4.0-86 of the /distribution/virt/install task for guest recipes has also been released:

  • #%s1180595: Updated to support the ppc64 architecture. (Contributed by Ján Stanček)
  • #%s1187266: Removed spurious RPM dependency on virt-manager, which is not available on all architectures and releases. (Contributed by Dan Callaghan)

A new core task, /distribution/command version 1.1-4, has also been released.