Release Notes

The following documentation provides release notes for individual versions of the Bowtie software. Release notes are broken down by the type of package (such as Client or Controller) and further grouped by type (such as bugfixes or features).

While we strive to provide complete release notes for updates, please be aware that these changes may not cover 100% of the changes included in a given release.

If you are looking for the latest version of a particular package, please rely on https://api.bowtie.works/ as the canonical source for latest updates.

Feeds

Release notes are also provided as RSS and Atom feeds:

• Atom feed

• RSS feed

Release History

25.08.002

Important

This release includes important changes that administrators should review.

Released on 26 August, 2025.

Client

Fixes

• Three related DNS changes in this release.

  1. A change in behavior was enacted to be defensive against consumer ISPs who use DNS interception techniques that overlap with private site ranges.

  2. All requests from the “Help” page to “Evaluate DNS Query” are now logged at $LOG_PATH/bowtie-dns.log and can be captured by support bundles. This can aid both administrators and Bowtie support in rapid analysis.

  3. The wording on that form has changed to be more clear for certain results that match user-supplied rules but are ultimately forwarded upstream.

• Disabled or deleted devices now correctly recognize their controller side state are able to re-authenticate users from the tray application.

• In prior versions of Bowtie, explicit routing exclusions and DNS routing exclusions were always routed through the default route. This may be incorrect in network scenarios with additional non-Bowtie routing table entries. With this fix, exclusions should now be routed to the correct destination.

• Linux: eliminated extraneous systemd-resolved restarts.

• Fixed an issue that could cause persistent, high background CPU use in some environments.

Enhancements

• Network change detection under Linux should be more reliable and incur less ambient CPU load.

Features

• Large Bowtie routes (any routes with a network mask of 255.0.0.0 or larger) are now removed from the routing table when a user does not have a connection to the internet (in the absence of a default route). This primarily affects users who are using Bowtie in a full tunnel configuration.

  To prevent this behavior, set the device configuration “controller-health-check-strategy” to “allow-presumptive-connections”.

• Extends device posture to Linux

• The network name is now displayed in the tray UI when Bowtie believes that the network is available. This network name is the adapter name on MacOS & Windows, and the SSID or a user friendly adapter name on Linux.

  This feature may be disabled by setting ui-network-status to false.

• Collect device posture (e.g. Bitlocker protection status) for Windows.

Server

Fixes

• The bowtie-server.service daemon should more reliably report its own version in output logs at startup.

Features

• Support bundles now record CPU use measurements.

• sshd listener options are now configurable in the Control Plane interface. Administrators may define an organization-wide default or per-Controller settings.

• opentelemetry-collector files are now configurable from the Control Plane web interface.

Enhancements

• The promtail.service log collection service has been removed in favor of the journald opentelemetry-collector-contrib receiver (documentation). promtail is deprecated and no longer receiving updates from Grafana Labs.

  If you rely on the default observability stack on Controllers to view logs with the default Grafana installation, no action is necessary.

  If you consume the log stream in a custom opentelemetry-collector-contrib configuration file, then you may want to ensure that exported logs continue to arrive in the expected format. Any custom log exporters defined in /etc/otel.yaml will still be deeply merged with the default configuration file, which has been updated with a new receiver and processor. The Bowtie documentation has been updated with the latest example of the default receivers and processors.

• The foundational operating system for Controllers has been updated from NixOS 24.11 to 25.05.

  This update includes the following noteworthy version changes. If you rely on any of these packages for downstream integrations (such as observability with Grafana and Loki), ensure that the updated versions are compatible with your existing configuration (and update those configurations if necessary).

  This change includes an upgrade from Linux kernel version 6.1.135 to 6.1.148. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

  Package	Old Version	New Version
  Linux kernel	6.1.135	6.1.148
  dex	2.41.1	2.42.0
  grafana	11.3.6	12.0.3
  loki	3.2.1	3.4.4
  prometheus	2.55.0	3.5.0
  python	3.12.8	3.12.10
  tempo	2.6.0	2.7.2
  opentelemetry-collector-contrib	0.112.0	0.124.0

  At time of writing, a scan of all operating system dependencies yielded no outstanding critical vulnerabilities requiring immediate action.

  The following updates may require manual intervention on the part of administrators:

  • jq has been removed from Controller images due to outstanding and unpatched vulnerabilities. Internal use of jq has been replaced with calls to thejaq executable (documentation here). If you rely on jq for custom scripts, consider migrating them to jaq, which is usually a drop-in replacement. For convenience, an alias to the jaq executable has been added.

  • opentelemetry-collector-contrib often includes significant changes between minor versions and may require updates to your /etc/otel.yaml file.

25.08.001

Important

This release includes important changes that administrators should review.

Released on 1 August, 2025.

Server

Fixes

• Fixed an issue that could prevent the Bowtie server process from coming online if it failed to find wireguard peers.

• Handle some edge cases to improve resiliency around provisioning new Controller certificates.

• Ensures that timestamps are correct when emitting logs from the Bowtie server daemon to OTLP endpoints.

• Improved the reliability for remote hostname and TLS checks during initial setup, particularly for deployments that only open port 443.

• Fixed an issue that caused nfqueue metrics collection to occur when no metrics were present.

• Fixed an issue that could cause updates to Bowtie server to fail to take effect when upgrading Controllers.

Features

• A new mechanism has been developed for automatic partition recovery. All controllers will retry all connections without intervention from the user.

  Two additional metrics of note are available in prometheus and opentelemetry:

  • bowtie_server_tcp_sync_manager_peers_we_have_count with label total should always match the number of active controllers.

  • bowtie_server_tcp_sync_manager_document_operations will grow over time, and will aid in observing the database size vs speed of other operations.

• Optionally send exceptions in the controller to Sentry for analysis. This feature defaults to OFF. If SENTRY_DSN is set then bowtie-server failures will be automatically emitted to your Sentry install. Additionally SENTRY_SAMPLE_RATE and SENTRY_TRACES_SAMPLE_RATE can be set to values between 0 and 10000 (100%) to emit additional metrics. Those also default to 0.

• Network plane activity now records user identifiers, device identifiers, and device hostnames when policy decision auditing is enabled.

• Some default behaviors of bowtie-server.service on Controllers have changed:

  • The OTLP_GRPC_ENDPOINT is now set to the local OTLP listener by default. This change serves to more tightly integrate the Bowtie daemon with observability tooling.

  • Audit-related events (such as packet flows, user authentication, and more) are now not emitted to standard output (and the system journal) by default. Due to their volume and potentially sensitive nature, they are now only omitted to OTLP endpoints and available over the logs signal.

  Operators currently relying on journal parsing to retrieve audit events should instead refer to the logs pipeline of opentelemetry-collector to optionally tap into the stream for other systems. By default, Controllers connect the OTLP log receiver to Loki, so audit events are still available for review but solely within Grafana and Loki rather than journald.

• Network plane activity now records source and destination ports when policy decision auditing is enabled.

• Policy verdict tracking may now be individually controlled between metrics and logs depending on Controller preferences.

• When policy metrics tracking is enabled for a Controller, these network policy decision events are now logged. To filter for these events, journal logs for bowtie-server should look for output with fields that match audit_event=true and audit_type=packet_queue.

• Network plane activity now records translated NAT64 addresses (if present) when policy decision auditing is enabled.

• Changes to the user-configurable opentelemetry-collector configuration file (/etc/otel.yaml) now trigger the opentelemetry-collector.service to restart.

• The otlp receiver for opentelemetry-collector.service has been added to the default services.pipelines.logs.receivers setting.

• Custom prometheus scrape configuration files may now be managed from the Control Plane interface.

• Certificates necessary for Controller SSO configuration files may now be uploaded from the Control Plane web interface.

• Improved the “validate hostname” step of the Controller guided setup process to more reliably complete.

Enhancements

• The default prometheus.service on Controllers now honors the user-controlled directory at /etc/scrape_config_files.d/*.yml to permit custom scraping endpoints.

• Third-party telemetry reporting for Grafana Loki has been disabled.

25.06.003

Released on 26 June, 2025.

Client

Fixes

• Fixed an issue where A records were returned for some AAAA queries when dns64_strategy was set as never.

• Linux: the behavior and appearance of the tray menu should more closely match that of Windows and MacOS.

25.06.002

Released on 17 June, 2025.

Server

Fixes

• In order to prevent a potentially ambiguous boot-up sequence: The configuration file /etc/bowtie-service.d/document-id.conf is specified as the preferred location for the BOWTIE_DOCUMENT_ID parameter.

• Fixed an issue with Incus images failing to boot by including EFI-enabled disk images.

Features

• Single sign-on (SSO) files may now be managed via the control plane web interface.

• Controller host load metrics are now included in support bundles.

• A new Grafana dashboard now ships with default Controller installations that visualizes access policy queue metrics.

• Marked Controllers images on GCP as GVNIC compatible.

• Controller support bundles now include limited certificate information as well as sanitized cluster membership information.

Client

Fixes

• When the external IP address of a Bowtie controller is within the range of a local non-default non-Bowtie network previous versions of the Bowtie client had trouble accessing that controller. This has been fixed.

• Connections over https tunnels now work better across network changes.

• Linux: fixed a bug in the network status indicator where disabled interfaces were not being properly ignored.

• Set the client status to “Initializing” if the block list is not loaded yet. Empty block lists are now handled correctly.

• In prior versions, if your authorization timed out you may not be able to access the login page without authorization.

Meta Control Plane

Fixes

• wget and curl download commands now wrap URLs in quotes to avoid breaking interactions with user shells.

25.06.001

Released on 2 June, 2025.

Client

Fixes

• All current nameservers can be added to our compatibility profile for DoH/DoT for private lookups. To enable this feature, use the new flag dns-block-doh-dot-current-nameservers = true.

• Status for DNS enforcing now includes waiting on the DNS block list to download and apply.

• Macos: In certain situations 25.05.001 through 25.05.003 preconfigured packages do not install the tray application.

Server

Features

• Single sign-on configuration files may now be uploaded as part of the guided web-based setup steps.

• Controllers now bundle the ncdu command-line disk usage utility.

• Controller images now support joining existing deployments with a guided, web-based setup without the need for cloud-init or manual intervention over the terminal via ssh.

Fixes

• Fixed an issue that prevented self-signed TLS termination on wildcard HTTP endpoints.

• Updated a reverse proxy service daemon setting to ensure that it never enters a permanently-down state.

25.05.002

Released on 29 May, 2025.

Server

Fixes

• During the initial setup phase, Controllers now flush setup keys to all available tty devices.

25.05.001

Released on 20 May, 2025.

Client

Fixes

• Flush the system-wide DNS cache after the DNS block list changes

• Fix the entrypoint dialog so clicking “Connect” works, in addition to pressing Enter on the text field.

• Fix a hang when setting the entrypoint in the configure dialog after a fresh install.

• Log errors gracefully when setting the tray icon.

• A compatibility mode has been added to allow docker containers in Linux to access private DNS.

• MacOS: A bug in Bowtie versions 25.02.001 through 25.03.003 has been fixed. Bowtie networks stopped working when the default route moves to another interface, such as when switching between Wi-Fi and Ethernet.

• In previous versions, the DNS filters may not have been available at client start. This has been corrected.

• Windows: A bug was found and fixed in the system that monitors the health of the Bowtie DNS system. This bug was introduced in 24.11.001 and fixed in 24.05.001.

Features

• A new setting is introduced for auditing client DNS events. Device config key dns-audit-level can be set to values “errors-only”, “blocked-queries”, “all-queries”, or “all-queries-all-answers” to log DNS queries and answers to a specified location on clients. The logs are written as JSONL following Elastic Common Schema.

  dns-audit-level is dynamic, it reloads on each DNS request. log-directory is not dynamic and takes effect only on client startup.

• Private nameserver lookups have improved performance. Additionally, there is a new DNS flag to shape behavior for upstream name servers. Lower timeouts can improve user experience for many cases so the default has been lowered. Additional changes to defaults may come as we collect more data in our environments.

  • private-upstream-timeout-ms: This was previously set at 5s and was not configurable. It has been reduced to 1.75s for the default.

• DNS audit logging now works on both Linux and Windows.

  This adds the setting dns_audit_log_directory. On Windows this defaults to a path under C:\ProgramData\Bowtie\log, on Linux this defaults to a path under /var/log/bowtie. This setting is only read at startup, it does not reload at runtime.

• Enable DNS audit logging on macOS.

• The configuration flag ui-network-status is added to control the network status line in the tray application, and is enabled by default.

• There is a new line in the tray menu reflecting network status. If this status line reports failure, it means the Internet access is unavailable.

• Support bundle packages now record the date and time they were collected.

Server

Fixes

• During database maintenance, a defect where device groups may be lost between versions 2025.01.001 and 2025.03.003 has been resolved.

• Cross Site DNS regressed in 25.03.003 clients with 25.03.003 controllers for some installations. A fix for these environments has been applied at the controller.

• Previously client configuration specific to user groups could be applied to accepted devices which were not yet associated to users. This has been resolved.

• In 25.03.003 an issue was observed where the DNS server on controllers could bind to an incorrect address. This is now checked and resolved at bowtie-service boot.

• Previously, services integral to BGP operation could enter a permanent-down state. New configuration changes ensure that attempts to restart an unhealthy BGP daemon will never fail permanently, but persistently retry with a moderate backoff.

  Note: BGP services may be inoperative but fail to broadcast network unavailability to other cluster peers, causing incorrect or unreachable routes. Investigative work into this failure mode is ongoing, but this change should mitigate some cases.

• Configuration files for Controller DNS are now included in Controller support bundles.

• Fixed an issue that could unintentionally cause systemd-networkd to restart on system update, potentially resulting in impacts to network connectivity.

• Fixed an issue related to AllocationUnits in .ova files that prevented them from being imported into certain VMWare environments.

• Fixed an issue that could cause reverse proxy configuration changes to fail on Controller update.

• Fixed an issue that prevented a system service (caddy-supervisor.service) from starting correctly.

• A regression preventing the telemetry preferences Control Plane page from rendering sample payloads has been resolved.

• Fixed a table styling issue on Controller initial setup pages.

Features

• Controllers now support the ability to create time-limited, serial console-only administrative user accounts for shell access from the Control Plane web interface. This feature is intended to aid administrators who may require host-level access without predefined access configured, such as via ssh. Consult the user documentation for temporary console users for additional information.

  Note: the addition of this feature does not create temporary administrative users by default, and may be disabled organization-wide if desired. Additionally, access is constrained to the equivalent of physical access to the controller. This feature does not enable remote access.

• Controller images are now available for Incus and LXD.

Enhancements

• Updated the base Controller appliance operating system to reflect the latest upstream package updates.

  The table below lists relevant user-facing packages that may impact custom configuration settings on deployed Controllers:

  Package	Old Version	New Version
  grafana	11.3.4	11.3.6
  linux	6.1.130	6.1.135
  nix	2.24.12	2.24.14
  caddy	2.9.1	2.10.0

  Vulnerabilities closed due to updated packages or backported patches include:

  Vulnerability	Package	Mitigation
  CVE-2024-56406	perl	Patch backported by upstream

  At the time of writing, a scan of all operating system dependencies yielded no outstanding critical vulnerabilities requiring immediate action.

  When upgrading your Controller, please bear the following release notes in mind:

  • This update includes an update from Linux kernel version 6.1.130 to 6.1.135. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

Documentation

Fixes

• Support for deploying Controllers on Kubernetes is undergoing frequent changes. Some sections of the documentation have been updated to reflect this.

  If your organization requires the ability to deploy on Kubernetes, please reach out to a member of our team, who will be able to assist.

25.03.003

Released on 31 March, 2025.

Client

Fixes

• In prior versions, locally scoped IPv6 upstream DNS servers were ignored. This has been resolved and clients having issues on networks that are primarily using DNS64 or other locally-scoped DNS servers have been repaired.

• The strategy for network interface configuration changes on Windows for the private Bowtie interface caused state tracking issues when connecting to many controllers at the same time, such as immediately after pause/resume cycles. The strategy has been adjusted to correct for this condition.

• Fix stderr redirection (only affects Windows) when the stderr file is deleted.

• Private DNS lookup now forward to the controller’s DNS server rather than directly accessing the upstream private DNS server. This reduces resource consumption.

Features

• Adds optional Sentry integration for error reporting. If the sentry-dsn is set, errors will be reported to Sentry. This allows for better monitoring and debugging of issues in production environments. It can further be controlled by sentry-sample-rate and sentry-traces-sample-rate. These are sent as integer values from 0 to 10000. They default to 100% (10000) for error samples and 10% (1000) for trace samples.

• WireGuard logs are split off into files with _wg to make the main logs easier to read.

Server

Features

• First-run installations for Controllers that undergo the /setup process now support loading pre-existing (bring-your-own) TLS certificates.

Enhancements

• Introduced additional safeguards around Controller REST endpoints. This is a preemptive defense in depth measure; control plane functionality should remain unaffected.

Fixes

• The sos command now asserts that the user has sufficient privileges to work correctly.

• Fixed an issue that could cause reverse proxy configuration updates to fail to apply correctly.

• Added additional measures to prompt retries when acquiring TLS certificates from ACME providers when failures occur.

25.03.002

Released on 18 March, 2025.

Server

Fixes

• A feature introduced in 25.03.001 to reset partial connections was shown to be unstable in some environments and is being disabled for further evaluation. Set WIREGUARD_ENABLE_KERNEL_LOGGING=true to enable.

• Fixed an issue when collecting BGP related logs from Controller SoS bundles.

• Added service manager settings that retry failed attempts at running dex.service.

• Updated the Controller command-line based support bundle utility to resolve deprecation warnings.

• Fixed an incorrect systemd setting for the backup daemon.

• Added service manager settings that retry failed attempts at running hostname-setup.service.

Features

• The bandwidth consumption of the Control Plane interface has been greatly reduced, now measuring roughly 1% of previous usage.

• The initial Controller setup page is now gated behind authentication. See the setup documentation for additional information.

Enhancements

• Updated the base Controller appliance operating system to reflect the latest upstream package updates.

  The table below lists relevant user-facing packages that may impact custom configuration settings on deployed Controllers:

  Package	Old Version	New Version
  grafana	11.3.2	11.3.4
  linux	6.1.123	6.1.130
  python	3.12.7	3.12.8
  git	2.47.0	2.47.2

  Vulnerabilities closed due to updated packages or backported patches include:

  Vulnerability	Package	Mitigation
  CVE-2024-52006	git	Updated to patched version
  CVE-2024-52005	git	Updated to patched version
  CVE-2024-50349	git	Updated to patched version
  CVE-2024-56171	libxml2	Updated to patched version
  CVE-2025-24928	libxml2	Updated to patched version
  CVE-2024-12797	openssl	Updated to patched version
  CVE-2024-9143	openssl	Updated to patched version
  CVE-2024-13176	openssl	Updated to patched version

  At the time of writing, a scan of all operating system dependencies yielded no outstanding critical vulnerabilities requiring immediate action.

  When upgrading your Controller, please bear the following release notes in mind:

  • This update includes an update from Linux kernel version 6.1.123 to 6.1.130. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

25.03.001

Released on 18 March, 2025.

Client

Fixes

• Allow the DNS service to run as a single process.

• Add an option to run the service as a Windows service directly, instead of using the winsw wrapper.

  See the changelog in crates/bowtie_service for details.

• Windows: logs are now limited to 50MB per file, with a maximum of 5 files per service.

• Obey the shutdown signal in the block list task of the main service.

  This improves graceful shutdown behavior but does not totally fix it, sometimes there is still a timeout.

• 25.03.001 changes the service management approach on windows.

  This will improve reliability at the point of upgrades by using native windows service control management and reduce an erroneous dialog in the MSI subsystem.

• In Windows, reading interface statistics on certain interfaces has occasional failures. Earlier versions of Bowtie reset the interface on a single read error. Bowtie now defaults to 3 consecutive read errors and this can be tuned with the interface-error-reset-count flag.

• Fixes the “The setup was unable to automatically close all requested applications.” dialog box on Windows.

• Bowtie will now reopen the login tab if a session expires or is forcibly disassociated.

• Close the tray application after StopServices in the MSI.

  If we close the tray application while the main service is still running, the main service’s UI supervisor feature may restart the tray application, causing the installer to see bowtie.exe as locked and prompt for a system reboot.

  Closing the tray application after the service is stopped should ensure that it stays closed during the update.

Features

• Toast notifications on Windows are now clickable.

• The default tunnel_rebuild_strategy is now RebuildOnAllUnreachable. This will reset the connection on certain connectivity issues.

• A new client configuration option has been added, allow-route-conflict-override.

  Bowtie does not install IP routes that shadow existing system routes. For example, if you have set up a Bowtie Site with a range of 192.168.5.0/24 but the user’s device already has a route of 192.168.4.0/22, Bowtie will not install the 192.168.5.0/24 route because that may break the user’s local network. Previously, the only way that users with this conflict could access the 192.168.5.0/24 site network was by using NAT64 translation.

  This new option allows this behavior to be overridden. In the above example, you can add 192.168.4.0/22 to allow-route-conflict-override to allow installation of the 192.168.5.0/24 route.

  This default value of this option is “10.0.0.0/8”. To return to previous behavior, set this client configuration option to an empty string.

  The format of this is a comma separated list of IP address ranges.

• Bowtie may now be configured to ask users for a reason when they pause. The user’s responses are collected and periodically uploaded to the controller, where they can be viewed by the Bowtie administrator.

Deprecations

• Previously Verisign and Hurricane Electric public DNS servers were configured as fallback options for both IPv4 and IPv6 to prevent certain local failure modes. Now dns_fallback_ipv6 will default to empty values, to prevent issues in split-horizon DNS environments.

Meta Control Plane

Fixes

• Updated the list of Windows and macOS versions to reflect more accurate client support coverage.

25.02.001

Important

This release includes important changes that administrators should review.

Released on 8 February, 2025.

Server

Features

• It has been observed that network partitions over a certain length (roughly 20 minutes) are not automatically recoverable and require a restart of some control plane components. To recover automatically in these scenarios via automatic server restart on peer failure, enable the “Minimum Peers Behavior” setting in a Controller’s settings in the Control Plane interface. By default, the setting is disabled to retain existing behavior and prevent spurious restarts.

  The “Minimum sync peers” field controls how many other Controllers the Controller being configured must be connected to before it is considered unhealthy. If using the setting, the number should be less than your cluster size, but greater than the number of Controllers at your site. This number is likely between 1 and 3 for most installations, and has a minimum value of 1.

  The “Sync-unhealthy duration” field controls how long (in seconds) it takes for the Controller being configured to restart after its number of connected peers falls below the set minimum. The minimum value of this field is 120 seconds.

  This setting can also be updated using the Controller REST API, with the /-net/api/v0/organization/controller POST endpoint.

  Note that updating the Minimum Peers Behavior setting for a given Controller will restart it.

  Further work is ongoing to make automatic recovery less disruptive and require less up-front consideration.

Enhancements

• Controllers now ship with a more comprehensive set of terminfo files to facilitate better client remote connection support.

• Controller updates now offer more fine-grained information regarding whether an update procedure either succeeds or experiences unexpected behavior.

• The foundational operating system for Controllers has been updated from NixOS 24.05 to 24.11.

  This update includes the following noteworthy version changes. If you rely on any of these packages for downstream integrations (such as observability with Grafana and Loki), ensure that the updated versions are compatible with your existing configuration (and update those configurations if necessary).

  This change includes an upgrade from Linux kernel version 6.1.119 to 6.1.123. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

  Package	Old Version	New Version
  Linux kernel	6.1.119	6.1.123
  cloud-init	24.1	24.2
  grafana	10.4.13	11.3.2
  loki	3.1.2	3.2.1
  opentelemetry-collector-contrib	0.101.0	0.112.0
  prometheus	2.54.1	2.55.0
  promtail	3.1.2	3.2.1
  python	3.11.10	3.12.7
  tempo	2.4.2	2.6.0

• Additional logging from systemd is now included in Controller support bundles.

Fixes

• Controller observability tooling was previously instrumenting all URLs, whether valid or not. Changes to the default Prometheus configuration now proactively drop URLs for paths with 404 responses to reduce noise and improve performance along with axum_http_requests_pending.

Client

Features

• Adds dns-capture-exclude-cidrs flag to allow bypassing bowtie-dns for one or more address ranges. This expands dns-capture-and-forward strategies to allow additional options for some software to pass bowtie unobstructed. CIDRs are delimited by a comma. Example: dns-capture-exclude-cidrs = "10.193.0.0/16,10.194.0.0/16"

• MacOS clients have now packaged wireguard-go as the default tunnel provider. Set wireguard-provider="Boringtun" to go back to prior functionality.

Fixes

• Improved behavior when installing or uninstalling Windows packages that should avoid notifications about needing to restart or failing to close the Bowtie client.

Meta Control Plane

Features

• The staging meta control plane (available for software downloads like release candidates) now has a more user-friendly URL at https://dev.api.bowtie.works available for general use.

25.01.001

Released on 9 January, 2025.

Client

Fixes

• Exit the tray app process cleanly when the main loop task ends.

  Previously on Windows, installing a new MSI would try to close the UI, but the UI would hang instead of exiting its process.

24.12.001

Released on 20 December, 2024.

Server

Fixes

• Network Interface reconciliation of routing tables and iptables rules previously only listened to changes coming from configuration. Now reconciliation events are considered from any change to netlink’s routing table, link state, or IP address assignments as well. Additionally reconciliation is clamped to also be evaluated periodically as a fallback.

Features

• Controllers now scale the nf_conntrack sysctl setting for optimal values. Previously, Controllers serving large volumes of traffic could exhibit session exhaustion resulting in dropped packet messages in some cases.

• User auditing events that occur over Bowtie tunnels now include additional address metadata about the public network address served by the tunneled interface.

Enhancements

• Updated the base Controller appliance operating system to reflect the latest upstream package updates.

  The table below lists relevant user-facing packages that may impact custom configuration settings on deployed Controllers:

  Package	Old Version	New Version
  grafana	10.4.8	10.4.13
  linux	6.1.111	6.1.119
  loki	3.1.1	3.1.2
  python	3.11.9	3.11.10

  Vulnerabilities closed due to updated packages or backported patches include:

  Vulnerability	Package	Mitigation
  CVE-2024-48958	libarchive	Updated to patched version
  CVE-2024-48957	libarchive	Updated to patched version
  CVE-2024-8006	libpcap	Updated to patched version
  CVE-2023-7256	libpcap	Updated to patched version

  At the time of writing, a scan of all operating system dependencies yielded no outstanding critical vulnerabilities requiring immediate action.

  When upgrading your Controller, please bear the following release notes in mind:

  • This update includes an update from Linux kernel version 6.1.111 to 6.1.117. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

• Additional system messages are included in Controller support bundles.

Client

Features

• On Windows, DNS policy enforcement now uses a more energy efficient interface

• On Windows, a new DNS supervisor strategy is implemented to improve performance and reduce energy use.

• Device to Controller Check-ins and Tunnel Statistics are now recorded to the local data store. Approximately 24h of active check-in responses are stored, and approximately 2 days of tunnel statistics are stored. Configuration defaults are:

  • store_maximum_checkin_response_rows=1440. This is approximately 24 hours of 1 minute check-ins

  • store_maximum_tunnel_stats_rows=1728000. This is approximately 2 days of 1 second stats for 10 controllers

• There is a new tunnel health checking mechanism. It uses a configurable persistent-keepalive on the tunnel interface to actively send zero-data length packets if the connection is quiet and bandwidth telemetry to passively determine tunnel health. The keepalive timer should match on the client and the controller. Enabling this feature requires environment variable configuration on every controller and client configuration on every client that you want to opt-in to tunnel rebuilds.

  This feature requires active data sent from the client and the controller. Persistent keepalives send a signed zero-length packet at a minimum interval on an otherwise quiet connection. You should set the persistent keepalives to the same value on clients and controllers.

  Set persistent keepalives on your controllers with this environment variable BOWTIE_WIREGUARD_PERSISTENT_KEEPALIVE=25

  Set persistent keepalives, and tunnel health failure boundaries on clients with these parameters:

  persistent-keepalive=25
  unhealthy-tunnel-health-count=10
  tunnel_rebuild_strategy=RebuildOnAllUnreachable
  

  This says “If telemetry counts are invalid for 25 * 10 seconds, then rebuild tunnels”

  Effectively, this will:

  • Rebuild tunnels on wake-from-sleep

  • Rebuild tunnels on implementation issues and severe network partitions

  It is our expectation that this feature will become stable and default soon when bandwidth and CPU costs of it’s operation decrease.

  There are two levels of enforcement. We currently recommend first trying tunnel_rebuild_strategy=RebuildOnAllUnreachable to opt into tunnel rebuild on failure.

• A new experimental routing provider for Windows that should substantially reduce energy consumption has been added. To enable, set the routing-provider to net-io-api. This will likely become the default in future releases.

Documentation

Enhancements

• Prevented support requests from being submitted twice at the same time and clean up the support tickets fields’ once submitted.

Meta Control Plane

Fixes

• Updated client package macOS compatibility to reflect that versions 14 and 15 are supported.

24.11.002

Released on 14 November, 2024.

Server

Fixes

• 24.11.001 introduced TCP support in the client DNS requests. 24.11.002 ensures that in all circumstances this traffic can transit the access policy engine.

Features

• NAT64 data flows multiplex over limited port resources on each controller. 24.11.002 tunes the size of the pool, as well as reduces the number of data flows that require NAT64.

24.11.001

Released on 12 November, 2024.

Client

Fixes

• Improvements for private name handling in IPv6-only networks when managed domains are in overlay mode.

• Fixed an issue in the Windows DNS supervisor causing errant service restarts when upstream connectivity is inconsistent.

• DNS health checks have also received several improvements. First, all health checks are now forced over the local upstream connection rather than allowed through the tunnel, which ensures a clear read of the client’s network environment. Second, the health check system is now respecting the probe’s TTL, which creates more accurate checks in full-tunnel scenarios and reduces noise on the network.

• Improvements in window handling in macos.

• Improved route handling for service and machine stop/start and wake/sleep events.

• Improved route handling for machines with multiple active network interfaces

Features

• Several improvements have been made to private name resolution. If multiple DNS servers are eligible, records will be requested in parallel, preferring the fastest result. EDNS and DNS over TCP are now enabled for private names, allowing very large records to be returned.

• SOS submissions may now be retried in case of errors to publish.

• Improved Active Directory integration for remote connections

• Important pre-release quality feature. controller-health-check-strategy. In previous Bowtie client versions, networks equal or greater to “/8” (e.g 10.0.0.0/8 or 0.0.0.0/1 for full tunnel) would not be installed until several health checks had passed. This is to prevent users from having “no internet” scenarios in captive portal situations. This flag can now be set to allow-presumptive-connections which will install the routes while Bowtie is Active, before health checks have passed. The next two versions will continue to improve on this strategy. If this flag causes issues, revert it to require-health-check to restore 24.10.003 default behavior. To manually bypass a captive portal while this flag is on, pause your Bowtie client. If this feature is right for your environment, consider client configuration targeting to deploy to a subset of your users.

Server

Fixes

• Increased the grace period that Controllers will allow when updating their BGP routes. Intermittent network latency fluctuations should result in reduced BGP configuration flapping.

• Fixed an issue that prevented Controllers from correctly provisioning locally self-signed certificates for local IP addresses.

Features

• If your public IPs are static, and if you always have public IP access to your controllers (via public access or hairpin NAT) a new Wireguard Hint field has been added to the cluster configuration. This will allow the clients to use the “Hint” IP for the Wireguard connection, instead of the DNS name. If you are connecting to your Bowtie controllers from the same network as your Bowtie clients, care must be taken to ensure that hairpin NAT is configured if they are using private IP addresses. If you are relying on split horizon DNS this implementation is not yet applicable to your environment.

Documentation

Enhancements

• Clarified the scope of the Control Plane devices permission as it applies to user information.

24.10.003

Released on 29 October, 2024.

Client

Fixes

• Updated the help menu item so it focuses the help window if it already exists.

• Previously server-sent configuration would not apply until after the first successful API command after boot. Now stored server-applied configuration is loaded at boot time.

Features

• When generating support bundles, users can now optionally add support details and contact information to their submission.

Server

Fixes

• Resolved an issue with BGP functionality that negatively impacted service availability on reload.

• Patched a bug in Controller clustering that would cause excessive traffic and log noise about peer members being known as new, different peers.

• Fixed an authentication route that could potentially expose an open redirect.

• Controllers now enforce stricter Set-Cookie settings.

• Closed an avenue for server information disclosure in the sos.service daemon.

• Grafana’s cookie security defaults have been hardened.

• Fixed broken links to collections when configuring web filtering.

Features

• The Control Plane settings interface now enforces correct values for web filtering collections if present.

• Controller support bundles now include more-complete wireguard interface information. Private keys are not included.

• If a Controller SoS fails to send correctly, the bundle payload is now cached for a short period of time. Administrators may optionally choose to retrieve this file to share out-of-band in situations like air-gapped Controllers or when the submission API endpoint is otherwise unavailable.

• The conntrack utility is now available on Controllers by default.

• Log out and log in buttons are now more prominently displayed on Control Plane pages.

24.10.002

Released on 10 October, 2024.

Client

Features

• In prefer-tunnel API mode we now return both IPv4 and IPv6 addresses for browser requests. This improves the user login experience when switching networks between on-site and Bowtie networks.

Fixes

• Fixes a regression in windows search domains.

24.10.001

Released on 9 October, 2024.

Client

Features

• Previously the re-authentication mechanism may prompt before it was possible (by network circumstance) to authenticate. This build checks for an “OK” response from the required endpoint before prompting the user.

Fixes

• In 24.09.007 an issue where the highest upgrade would be re-applied has been resolved.

Server

Fixes

• In 24.09.007 gossiped ephemeral messages between nodes in large clusters could pass a size boundary which would crash the node. These messages are now dropped and logged.

• Controllers would sometimes run the zebra.service daemon even when BGP was not enabled. This service now only runs when necessary.

• Fixed an issue in which mgmtd-config.service or bgpd-config.service units might have failed during Controller upgrade.

• The /sos HTTP endpoint now denies access unless the request originates from a logged-in user. If you need public access for Controller SoS bundles, consider using the Control Plane support page, the port :911 HTTP endpoint, or the sos terminal command.

Features

• A grace period can be configured for user authentication sessions. on the /configuration page if you have user device disassociation time set you may also set a grace period. For example if you set the timer to 12 hours, and the grace period to 1 hour, the user will be prompted starting at hour 11 after authentication, but the policy engine will not disassociate the user and the device until hour 12.

• Access policy performance for TCP flows has increased.

• Logging verbosity around device<->user binding and user authorization has increased. All events regarding this are labeled audit_event=true

• Logs related to supporting BGP daemons are now included in Controller support bundles.

• Controller gce and gce-efi images now include google-cloud-sdk.

Meta Control Plane

Features

• A new field named commentary is now present on reported vulnerabilities. If set, the contents of the field explain why the vulnerability presents a reduced risk to the given software package.

24.09.007

Released on 27 September, 2024.

Server

Features

• Add the ability to specify next_hop for any site’s range.

• Controllers now emit all user authentication events in their server logs. When viewing bowtie-server.service logs on the Controller directly, via Grafana, or aggregated through opentelemetry-collector, these auditing log events are annotated with audit_event=true. You may use this field to narrow searches for user authentication activity in the log event stream.

  The types of auditing events include successful authentication, denied authentication (through invalid credentials or disabled users), and initial authentication flow requests. Wherever possible, these events include source IP metadata derived from the best possible source (deferring to IP forward headers and falling back to direct peer network address).

• Controller support bundles now accept an optional contact and problem description field.

• Controllers now make python available in-$PATH for use with tools that require an interpreter like ansible. At time of writing, the bundled python version is 3.11.9.

Enhancements

• Updated the base Controller appliance operating system to reflect the latest upstream package updates.

  The table below lists relevant user-facing packages that may impact custom configuration settings on deployed Controllers:

  Package	Old Version	New Version
  bash	5.2p26	5.2p32
  grafana	10.4.6	10.4.8
  linux	6.1.104	6.1.111
  prometheus	2.53.1	2.54.1

  Vulnerabilities closed due to updated packages or backported patches include:

  Vulnerability	Package	Mitigation
  CVE-2024-41815	starship	Backported patch
  CVE-2023-42366	busybox	Backported patch
  CVE-2023-42365	busybox	Backported patch
  CVE-2023-42364	busybox	Backported patch
  CVE-2023-42363	busybox	Backported patch

  At the time of writing, a scan of all operating system dependencies yielded no outstanding critical vulnerabilities requiring immediate action.

  When upgrading your Controller, please bear the following release notes in mind:

  • This update includes an update from Linux kernel version 6.1.104 to 6.1.111. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

Fixes

• Set a cloud-init option that should avoid losing manual address assignments on network interfaces via DHCP.

• Enabled the Controller mgmtd daemon to facilitate multi-hop routing when required.

Client

Features

• To provide enhanced device security, Bowtie has begun storing device-specific secrets in the operating system’s secret storage mechanism instead of privileged files on the operating system. Retrieval and use of those secrets is currently locked behind a feature flag. Set --state-strategy=LoadFromStateDb to prefer the SQLite DB and OS Secret storage mechanism. Future releases will remove existing privileged files and default to OS Secret storage.

• API communications over the Bowtie tunnel are now supported. Previously both TCP443, for API communications and a UDP port for tunnel communications were required. Bowtie will now allow TCP connections over the tunneled connection. This will allow in many scenarios a greatly reduced public surface area of your secure infrastructure. This mode of operations currently requires bootstrapping each device with accessible HTTPS connections either via a privileged connection (like in-office) or temporary access over the tunnel (like allowing a specific device through your firewall for TCP443 for a temporary amount of time). Future iterations may allow easier device bootstrapping through alternate channels. If you are interested in other modes of operation please discuss it with your Bowtie representative. To enable this functionality set controller-api-strategy=prefer-tunnel.

• The software-update-strategy configuration variable gained a new value: auto-managed-in-range(min_version, max_version)

  With this strategy, updates are managed by Bowtie, but only within a specific version range (inclusive). If the current version is within the range, no updates are performed. If the current version is outside the range, an update will be performed to the highest available value within the range. Example configuration file entry: software-update-strategy = "auto-managed-in-range(24.09.006, 24.09.008)".

  There are 4 new auto-update configuration variables:

  software-update-time-start: The start of the time window within which auto-updates are allowed to be installed. If software-update-time-start is greater than software-update-time-end, the window includes midnight. If software-update-time-start is equal to software-update-time-end, the window is all 24 hours. Default: midnight. Time format: “HH:MM:SS”, where HH is 00-23. Times are specified in local time.

  software-update-time-end: The end of the time window within which auto-updates are allowed to be installed. If software-update-time-start is greater than software-update-time-end, the window includes midnight. If software-update-time-start is equal to software-update-time-end, the window is all 24 hours. Default: midnight. Time format: “HH:MM:SS”, where HH is 00-23. Times are specified in local time.

  software-update-interval: How long to wait between auto-update checks. May be specified like “10d”, “10h”, “10s”. Quotes are necessary in TOML configuration files. Default: “1d”.

  software-update-delay: How long to wait after starting Bowtie before the first auto-update check. May be specified like “10d”, “10h”, “10s”. Quotes are necessary in TOML configuration files. Default: “1h”.

Fixes

• Windows & Linux: downloaded auto-update packages are deleted immediately after use rather than relying on the operating system’s temporary directory cleanup mechanism.

Meta Control Plane

Features

• Vulnerability reports for each Controller artifact are now reported alongside other download metadata. Consult the vulnerability API documentation for additional information.

  Scans are performed on a regular basis for current Controller versions.

24.09.006

Released on 17 September, 2024.

There are no release notes for this version.

24.09.005

Released on 13 September, 2024.

Server

Fixes

• For LDAP Sync Jobs: A defect in our pagination approach was preventing any results from being returned for large queries of Groups. The output of the LDAP sync task has been truncated to report statistics and sample the first result for additions.

• Fixed an issue that would cause the backupd backup daemon to try and fail to start indefinitely on pre-bootstrapped Controllers.

Features

• Controllers now print their network interface information to the serial console at boot time to aid with operational diagnostics.

Client

Features

• A new configuration option for MacOS can aid in co-existence with other networking software that uses pf. If you have custom anchors in pf you can add anchor-excludes=someanchor,otheranchor and the Bowtie client will ignore those rules when adding its own.

Fixes

• A routing calculation defect was introduced and identified in MacOS on 24.09.004.

• Bowtie works hard to play nicely with other networking software on your computer. Our persistence mechanism on MacOS for making sure other software plays well with us has proven to be stable and effective in a large variety of circumstances. The heartbeat mechanism for this check has been reduced from 2 minutes to 30 seconds.

24.09.004

Released on 10 September, 2024.

Server

Fixes

• We have resolved an issue where incomplete configurations of BGP mode could leave a controller in a crash loop.

• Fixed some cases in which updates would cause the Control Plane UI to fail to load.

Features

• The IPv4 Pools page has changed to make selecting routing strategy decisions easier to understand, and explicitly listing every available option.

24.09.003

Released on 5 September, 2024.

Client

Fixes

• In 24.09.002 on MacOS Bowtie DNS would not be functional concurrently with the Operating System’s “Internet Sharing” feature. This is resolved in 24.09.003

24.09.002

Released on 5 September, 2024.

Server

Features

• Up to this point, Bowtie has only allowed routing via NAT through our controllers. Significant changes have been made to allow preview release of highly-available direct routing via BGP participation at your sites.

  With this change, a previously required kernel module is not necessary in all circumstances. --enable_nat64_kernel_module=false can now be used on controllers where no NAT64 capability is necessary.

• Controllers now have the ability to selectively block IP addresses or CIDR networks from control plane web access. Reference the web filter documentation for additional information.

Enhancements

• Updated the base Controller appliance operating system to reflect the latest upstream package updates.

  The table below lists relevant user-facing packages that may impact custom configuration settings on deployed Controllers:

  Package	Old Version	New Version
  grafana	10.4.4	10.4.6
  grafana-loki	3.0.0	3.1.1
  linux	6.1.96	6.1.104
  prometheus	2.52.0	2.53.1
  promtail	3.0.0	3.1.1

  When upgrading your Controller, please bear the following release notes in mind:

  • This update includes an update from Linux kernel version 6.1.96 to 6.1.104. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

  At the time of writing, a scan of all operating system dependencies yielded no outstanding critical vulnerabilities requiring immediate action.

Fixes

• Improved Content-Security-Policy handling by proactively caching assets.

• Addressed an issue that could cause Control Plane web pages to present Content-Security-Policy loading errors resulting in an empty white screen.

• The Controller Control Plane web interface now has more targeted caching headers set to ensure that the correct assets are used. This bug would manifest as web interface bugs following a Controller version update.

• The qemu guest agent now runs on all qcow and qcow-efi formats (conditionally upon whether the host is running inside of qemu).

Client

Features

• A mechanism to limit CPU usage was added.

  This also should help limit runaway logs.

Fixes

• In Version 24.09.001 a regression occurred in allowing private names to be resolved with some DNS configurations.

24.08.003

Released on 9 August, 2024.

Server

Features

• Device limit counts are able to be set per user and globally. Users will be redirected to /device-auth-limit-reached if they log in to a device which exceeds their authorization level.

• New features have been added to the Configuration section to remove or disassociate stale devices which have not checked in over some time (greater than one week). If those options are set then devices will be scanned hourly.

  Additionally broadcast, and cache device check-in data between all controllers so that more accurate “last seen” information can be found in the API.

• Controllers now support in-browser chat-based support options for licensed customers. Refer to the support chat documentation for additional information about this feature.

Client

Features

• Added support for Ubuntu 20.04.

• “kebab-case” is now allowed in client configuration files.

  Prior to this update, configuration files contained lines like

  dns_capture_and_forward_strategy = "AllowAlternatePrivateF1"
  

  Now, “kebab-case” is allowed for both key and value:

  dns-capture-and-forward-strategy = "allow-alternate-private-f1"
  

  The old casing is still accepted but the documentation will recommend “kebab-case”, primarily due to its similarity to the usage of the same parameter on the on command line:

  $ bowtie-service --dns-capture-and-forward-strategy=allow-alternate-private-f1 service
  

Fixes

• IPV6 fix for dns_capture_and_forward_strategy = "AllowAlternatePrivateZ1AndS1"

24.08.002

Released on 2 August, 2024.

Client

Fixes

• Version 24.08.001 was released with an executable component on windows that was not signed by our CI. This package has been pulled and this is resolved in 24.08.002

24.08.001

Released on 2 August, 2024.

Client

Features

• In order to mitigate caching concerns, upstream health checks have migrated to a payload-based approach.

• MacOS: a new strategy was added to allow coexistence with an alternate VPN solution.

  Enable with dns_capture_and_forward_strategy = "AllowAlternatePrivateZ1AndS1" in your configuration file.

Server

Fixes

• Version 24.07.005 included a change to open the Controller’s reverse proxy to respond to any inbound request with TLS (regardless of hostname). This change has been made more consistent to also respond with a self-signed certificate (if necessary) to requests from any IP address as well.

24.07.007

Released on 26 July, 2024.

Client

Features

• Three new options have been added to the DNS supervisor to modify the behavior of the DNS supervisor on Windows.

  // These are the defaults
  dns_healthcheck_failure_uninstall_count: 3,
  dns_healthcheck_local_timeout: 150, # Milliseconds
  dns_healthcheck_upstream_timeout: 500, # Milliseconds
  

  The previous default for dns_healthcheck_failure_uninstall count was 1. In low-quality internet scenarios increasing this counter may help keep connections stable. The other behaviors have not changed by default.

• Windows: NrptRules has been made the default DNS injection strategy. To restore the previous behavior, add to your configuration file in C:\ProgramData\Bowtie\configuration\ the line local_dns_provider = "RegistryInjector".

Fixes

• Red Hat / CentOS Linux: fixed errors in the uninstall and upgrade scripts for 24.06.006. To uninstall 24.06.006 completely, upgrade to 24.06.007 and then uninstall. Upgrades from 24.06.006 to 24.06.007 will not take effect until after a reboot or manual restart.

• Bowtie will now try two different strategies for Wireguard endpoint address lookup. This may help when switching between networks where the endpoint has a different IP address on the two networks.

24.07.006

Released on 18 July, 2024.

Server

Fixes

• Some control plane API routes have been restricted to logged-in users only.

• Fixed an issue that precluded the sos daemon from starting directly in some cases.

Features

• A new task to clean up orphaned KVS files, like downloaded network block lists, has been created. By default it runs every 4 hours.

Client

Features

• Bowtie client rpm packages are now available for CentOS Stream. See the client setup documentation for additional information about using these new packages.

Fixes

• Windows: handshake status now displays correctly.

24.07.005

Released on 16 July, 2024.

Client

Fixes

• Fixed an issue on Windows where Bowtie DNS would sometimes fail to pick up DNS server changes.

• Windows: if Bowtie DNS fails health checks for an extended period of time, it is restarted.

• The support page now populates its status and organization detail information faster.

Server

Fixes

• Fixed an issue on Controller xen images that caused errors related to grub installation when undergoing updates.

• Controller HTTP and HTTPS endpoints will now respond to the backend health check path (/-net/api/v0/ok) in response to any request regardless of Host header.

• Bowtie Controllers are now identified by their appliance name and version in ESXi environments.

Features

• Typeahead search has been added for policy building and user group management.

24.07.004

Released on 10 July, 2024.

Server

Fixes

• A defect has been identified under certain circumstances when adding and removing controllers via the API. This release includes a fix for a subsequent failure to boot controllers after the defect case has occurred.

24.07.003

Released on 9 July, 2024.

Client

Features

• Bowtie will now always ensure there is a backup of the configuration file.

Fixes

• There are new mechanisms for recovering from failed states on the client. The Bowtie client now stores multiple copies of the critical information needed to operate clients on disk. In the event of data loss from disk degradation or program errors in one area, many circumstances can now be fully recovered.

• On Windows, the light and dark mode tray icons should now look a bit nicer with some display scaling settings.

• Linux: if you originally installed Bowtie with a .deb package that contains a configuration file and upgrade with a .deb package that does not contain a configuration file, the configuration file will be restored from backup.

Deprecations

• Bowtie will now only read configuration files with the extension .conf or .toml

Server

Enhancements

• The Controller reverse proxy software has been updated to Caddy 2.8.

24.07.002

Released on 5 July, 2024.

Client

Fixes

• Version 24.06.003 introduced DNS health checking. One endpoint/check pair was dns-upstream-test.bowtie.works pointed at TEST-NET-3 addresses in RFC1918 space. This was found to be filtered by some DNS providers, erroneously suggesting that DNS was failing and Bowtie would refuse to install DNS. Now two addresses are checked for static upstream responses: dns9.quad9.net & dns.msftncsi.com

Features

• On Windows, the Bowtie tray icon will now automatically change its appearance when the Windows color mode setting is changed between dark and light.

Server

Fixes

• Errors in task execution would sometimes cause a panic. They now are handled correctly.

• Controller backup and restore operations were not capturing reverse proxy material like Caddy configuration and TLS certificates. This update improves the backup service to include these services and files.

Features

• A new Web GUI has shipped. Includes several new features including better live input validation in many areas. It’s also quite good looking.

24.07.001

Important

This release includes important changes that administrators should review.

Released on 1 July, 2024.

Server

Fixes

• Updated Controllers to a patched version of openssh to address CVE-2024-6387. The service will restart automatically on update; no further action is required after performing a Controller update.

Meta Control Plane

Fixes

• Fixed a minor bug that listed some release candidate versions in the incorrect order when querying for the latest package.

24.06.003

Important

This release includes important changes that administrators should review.

Released on 27 June, 2024.

Client

Features

• Adds a local_dns_provider="NrptRulesSplitDns" which obviates “No Internet” reported issue on windows and helps connection stability in many circumstances, only sending requests to Bowtie DNS for Managed Domains

• Adds a local_dns_provider="NrptRules" which obviates “No Internet” reported issue on windows and helps connection stability in many circumstances.

• A new configuration option auth_prompt_strategy=Never has been added which is appropriate for device-only authorization and multi user terminals.

Fixes

• We have added several checks to configuration file loading to provide more information and smoother operations if any configuration file is damaged or incomplete.

• A new state machine has been added for DNS supervision. Bowtie will only take over primary DNS if all required checks pass, and will uninstall itself if any tests fail.

• Addressed some cases where the Bowtie GUI application (tray icon) would fail to launch. It should now always launch successfully following an interactive installation.

• The wireguard interface will now be downed when a pause is requested.

Server

Enhancements

• Controller SoS bundles now include additional information about configuration settings like backups and automatic updates.

• The foundational operating system for Controllers has been updated from NixOS 23.11 to 24.05.

  This update includes the following noteworthy version changes. If you rely on any of these packages for downstream integrations (such as observability with Grafana and Loki), ensure that the updated versions are compatible with your existing configuration (and update those configurations if necessary).

  This change includes an upgrade from Linux kernel version 6.1.82 to 6.1.95. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

  Package	Old Version	New Version
  Linux kernel	6.1.82	6.1.95
  cloud-init	23.1.2	23.3.3
  amazon-ssm-agent	3.2.1798.0	3.3.484.0
  awscli2	2.13.33	2.15.43
  cloud-init	23.3.3	24.1
  grafana	10.2.6	10.4.4
  loki	2.9.6	3.0.0
  openssh	9.6p1	9.7p1
  opentelemetry-collector-contrib	0.87.0	0.101.0
  prometheus	2.49.1	2.52.0
  promtail	2.9.6	3.0.0
  python	3.11.8	3.11.9
  tempo	2.3.0	2.4.2

Fixes

• Fixed an issue that caused the service that printed OpenSSH host keys at start time to crash when it encountered DSA keys.

• Fixed an issue that would cause Grafana OAuth authentication to enter an infinite location redirect loop.

• Fixed an issue that prevented user edits from being saved.

24.06.002

Released on 21 June, 2024.

Client

Fixes

• On Windows: At boot, awake from sleep, or certain network change events, there was a race condition between bowtie-dns and the network interface coming online. An upstream test was added and the service is much more resilient to network change events of any kind.

• Bowtie refuses to install routes that will cause conflicts with already existing system routes. This logic was too aggressive, and now Bowtie will install routes that overlap but do no conflict.

  For example, if you have a system route 10.0.0.0/24 and wish to install a Bowtie route at 10.0.0.0/8 this will now work. The 10.0.0.0/24 route is more specific and the OS will preferentially route to it. The Bowtie route will not interfere with the existing system route.

• Bowtie will now not switch networks from one wireguard peer to another as long as the previous peer is still up.

• The Bowtie installer now deletes any log files created by older versions of Bowtie that used different log file names.

• A MacOS segfault introduced in 24.05.003 has been fixed.

Features

• On MacOS Bowtie now creates a _bowtie system user and uses it to run the DNS daemon. (Linux has used the bowtie user since 24.05.001)

• Wireguard handshake status added to support help page.

Server

Features

• Controllers now support the ability to perform regularly-scheduled backups and restore from those backups either interactively or automatically. Consult the backup and restore documentation for additional information.

24.06.001

Released on 6 June, 2024.

Client

Fixes

• A memory leak was fixed in Linux

Features

• Four different strategies for setting Linux search domains have been added. The configuration variable search-domains-strategy may be set to one of these 5 values:

  • none

  • resolvectl: use the resolvectl CLI tool

  • resolved-conf-d: stores search domains in /etc/systemd/resolved.conf.d/bowtie-search-domains.conf

  • network-manager-conf-d: stores search domains in /etc/NetworkManager/conf.d/bowtie-search-domains.conf

  • network-manager-dbus: notifies NetworkManager about the search domains using D-Bus.

24.05.005

Released on 31 May, 2024.

Server

Fixes

• Fixes an issue in which IPv4 address allocations where not reused when already assigned to a device. If a device contains an IPv4 allocation then it’s reused and assigned appropriately.

• Optimized Controller images to reduce their overall system size.

• Updated the version of git included on Controllers to 2.44.1 to address recent CVEs.

• The user edit form in the administrative panel has been modified so that feedback on password requirements is shared, and errors prevent the form from closing and resetting.

Features

• Controllers now bundle Zabbix agent 2. The daemon is not enabled by default and must be configured before use. Consult the related documentation for additional information.

Client

Features

• The Bowtie client has gained the API endpoints: http://localhost:17133/organizations, http://localhost:17133/organizations/:id and http://localhost:17133/organizations/:id/peers

  The latter two endpoints support Content-Type: text/event-stream as well as Content-Type: application/json

  These endpoints will be used by the Bowtie client support page which is accessible via http://localhost:17133/static/support.html or help in the tray menu.

• The help page (also available at http://localhost:17133/static/support.html), has gained more information, including detailed information on wireguard peers.

Fixes

• Fixed an issue where check-ins returning HTTP 204 were reporting as HTTP 400 on the status screen.

24.05.004

Released on 23 May, 2024.

Client

Fixes

• Version 24.05.003 reported as 24.05.003-rc.6. Version 24.05.004 is being released to eliminate ambiguity between released and unreleased software. There are no functional changes between 24.05.003 and 24.05.004. Incorrect tagging affected client-reported version numbers in the devices page and API and reported version number on the support window of the tray application.

24.05.003

Released on 22 May, 2024.

Server

Fixes

• The bowtie-server process on Controllers was not honoring all signals to shut down cleanly, which it now does.

Features

• Support bundles emitted from Controllers now include iptables dumps.

Client

Features

• Adds two local DNS observability feature, both accessible via the support page from the tray icon. The first is a dig-like resolving tool to explain private tunnel DNS modifications. The second is an upstream monitor to show what Bowtie DNS will forward to for other internet traffic. This record is pulled from your local operating system, and when that is not available fallback records can be set in your configuration file at dns_fallback_ipv4 and dns_fallback_ipv6. Fallback records currently default to public DNS providers at Hurricane Electric and Verisign.

  Additionally DNS upstream change resolution time has been reduced from 15-30s to a median time of 1.5s for network change events on Windows. MacOS and Linux upstream resolution continues to be effectively immediate, querying the operating system on every request.

Fixes

• DNS concierge & block lists are now disabled while Bowtie is paused.

24.05.002

Released on 15 May, 2024.

Server

Fixes

• 24.05.001 included a regression for IPv4 pools that did not persist through controller restarts. This has been resolved.

24.05.001

Released on 13 May, 2024.

Server

Fixes

• Fixes an issue in which new IPv4 pools assigned through the web client were not assigned to the wireguard interface. Previously a controller restart was required to resolve the issue. Modifications to IPv4 pools are now effective immediately and no longer require controller intervention.

Features

• tmux is now included on all Controllers.

• IPv4 Address Pools delegated to Bowtie for assignment can be deleted, with cascading effects from the administrative web UI. They can also be deleted from the API by setting cascade=true.

Client

Features

• A compatibility mode was added to MacOS for services which scan local TLS traffic.

Fixes

• On Windows, updating to a new version of Bowtie will now remove any previous versions already installed. It should no longer be possible to have more than one version of Bowtie installed at a time.

• Silent installations on Windows (including automatic updates where those are available) will now launch the Bowtie tray icon, just like interactive installs can.

24.04.004

Released on 29 April, 2024.

Client

Features

• On Windows, Bowtie now installs two separate services, which you can see as Bowtie Service and Bowtie DNS in the Windows services utility. This should enable faster responses to network changes.

• On MacOS and Linux, Bowtie has gained the ability to run its DNS component as a user other than root. This can be enabled by either setting the dns_user_id setting in the configuration file, or by creating a user named bowtie.

  The .deb install package creates a bowtie user, so it will run DNS as a non-root user by default. The same capability exists in MacOS but it is not yet turned on by default.

• When you request a support bundle from the Bowtie client, that will be automatically sent to Bowtie support. You can opt-out of this behavior by setting send_sos_to_bowtie_strategy = "OnDemand" in configuration.

Fixes

• The default log level for the Bowtie client has been set to info, up from debug. To revert this change you can set verbose = 1 in the Bowtie configuration file. Further reducing the log level ton warn can be done with silent = 1; to error with silent = 2.

• Windows 10 clients should no longer see the “No Internet” status reported for Internet-connected networks after e.g. resuming form standby.

• The default for metrics_socket_address has changed from the Wireguard IPv6 address to 127.0.0.1:17133.

• On MacOS, Bowtie temporarily disables during a captive portal entry session to allow the portal to be accessed.

Deprecations

• The .exe Windows package has now been fully deprecated in favor of the .msi package.

Server

Features

• Assisted support (SoS) capabilities for Controllers are now available in the Control Plane UI on Controllers and as the sos command on the command line.

• Controller SoS endpoints now support browser-based interactions into addition to plain API-based requests.

• Support bundles (SoS diagnostics) may now be generated on Controllers exclusively via the command line instead of solely over HTTPS.

Enhancements

• The control plane web interface administrative configuration section now includes the ability to change telemetry preferences.

24.04.003

Released on 15 April, 2024.

Client

Features

• A compatibility mode was added to MacOS for services which exclusively use local private IP address resolution.

24.04.002

Released on 11 April, 2024.

Server

Fixes

• Fixed a bug that caused errors when provisioning ACME TLS after a problematic Caddy update. TLS provisioning should be functional once more.

24.04.001

Released on 9 April, 2024.

Server

Enhancements

• The /grafana Controller endpoint is now access-controlled via login to the Controller control plane web application. To continue accessing Grafana, please ensure that you are logged in to the Controller’s web interface.

• Updated the base Controller appliance operating system to reflect the latest upstream package updates.

  The table below lists relevant user-facing packages that may impact custom configuration settings on deployed Controllers:

  Package	Old Version	New Version
  dex	2.38.0	2.39.0
  grafana	10.2.4	10.2.6
  grafana-loki	2.9.4	2.9.6
  linux	6.1.78	6.1.82
  prometheus	2.49.0	2.49.1
  promtail	2.9.4	2.9.6

  When upgrading your Controller, please bear the following release notes in mind:

  • This update includes an update from Linux kernel version 6.1.78 to 6.1.82. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

  • Dex has been updated from version 2.38.0 to 2.39.0. Upstream release notes include important changelogs about LDAP connectors.

  At the time of writing, a scan of all operating system dependencies yielded no outstanding critical vulnerabilities requiring immediate action.

Fixes

• Raised the default web frontend rate limit significantly to avoid spurious HTTP 429 errors. This setting remains configurable if issues persist in particular environments.

24.03.007

Released on 29 March, 2024.

Server

Features

• policy objects now contain a status field that can be in either an enabled/disabled state. If a policy is disabled it’s evaluation will be ignored. Enabled is the default status to maintain backward compatibility of the API, and existing values in the database. The UI has been been modified to allow for a user to toggle the status of one or many policies.

• Individual IPv4 Addresses can now be released via the API or Web GUI.

• Controller images are now available in Azure GovCloud. See the Bowtie downloads page for Azure for a list of regions including those hosted in GovCloud.

• Controllers now support the ability to collect and deliver diagnostics bundles to Bowtie for assisted support efforts in the event of a troubleshooting situation. Data is not sent by default and explicit administrative steps are necessary to initiate the process. See the Controller SoS section for additional information.

Fixes

• Fixed an issue where the single sign-on service provider (dex.service) would fail on Controller startup or update when no configuration files were present.

Client

Fixes

• On versions prior to 24.03.007, system DNS could be broken by certain combinations of WiFi network changes and switching from wired Ethernet.

Documentation

Features

• Added Windows client support for DNS search domains.

  Concierge domains configured as search domains in the Bowtie server will now function as search domains on clients. This means that for instance, for concierge domain example.com configured as a search domain, the name app1 will now resolve as if the user had entered app1.example.com, and typing just app1 into e.g. a web browser should work as expected.

  Currently, this feature is only present on Windows clients.

24.03.006

Released on 20 March, 2024.

There are no release notes for this version.

24.03.005

Released on 19 March, 2024.

Client

Fixes

• In windows a crash loop was observed in 24.03.004 preventing normal operation for most users. This was due to a change in our IPC passing and window handling code. The crash loop has been resolved.

Server

Features

• Previously, the smallest unit of a policy destination was (“host”, “protocol”, “port”); with no way to compose arbitrary host groups. We found this to be limiting to creating narrow rule sets. We have introduced host collections to alleviate this. It is currently in an early release state and API-only. If your rule set would benefit from host collections, please reach out and watch this space.

24.03.004

Released on 12 March, 2024.

Server

Fixes

• The OpenAPI spec for the Bowtie server API now contains a more complete listing of available API endpoints and associated types.

• Navigating between tabs in the Settings page should be a smoother experience.

• A component for displaying policy resources was regularly reused. This created some confusion with which actions were available in which areas. The “Trash” icon no longer shows up on policy resources in read-only contexts.

• Fixed an issue that prevented API keys from being deleted.

• Fixed CSS rules that were preventing some fields from displaying legibly when browsing in dark mode.

Features

• The User Groups page now allows group deletion.

Documentation

Features

• Added documentation for address pools.

• Added documentation regarding how to configure and enable automatic Controller updates via the control plane web interface.

24.03.003

Released on 7 March, 2024.

Client

Fixes

• Versions 24.02.004 through 24.03.002 of bowtie-dns returned IPv4 A records for private resources that were inaccessible over IPv4 if no IPv4 addresses were configured.

24.03.002

Released on 6 March, 2024.

Server

Features

• Controller images are now available for Xen-based hypervisors including XCP-ng. See the Bowtie download platforms page for Xen to download .ova appliance image files.

24.03.001

Released on 6 March, 2024.

Server

Fixes

• Active Directory GUIDs were being ingested with inconsistent formatting. The binary representation via LDAP was different from the hex representation via OIDC. This is now repaired in the LDAP sync task.

Enhancements

• ESXi .ova image default resource allocations have been adjusted to reflect recommended system requirements (4 cores and 4GB memory).

Features

• Auto-updates per-controller can now be configured by clicking on the controller in the /control-plane view in the web GUI.

• IPv4 pools with manual address assignment are now able to be assigned from the Devices page in the web GUI.

24.02.006

Released on 29 February, 2024.

There are no release notes for this version.

24.02.004

Released on 29 February, 2024.

Client

Fixes

• At 24.01.001 an upgrade of a dependency caused changes in behavior on MacOS which meant the Bowtie icon appeared in the task switcher and the dock. This has been resolved.

Known Issues

• There is a known issue where the Bowtie tray application sometimes fails to start successfully on Windows. The Bowtie service will still be running, you may continue without the tray application. You can restart the tray application by searching for “Bowtie” in the windows menu, by clicking on C:\Program Files\Bowtie\24.02.004\bin\bowtie.exe or by restarting your computer.

Features

• When IPv4 Support is enabled on the controller for a given device, A records are no longer dropped by default. DNS Strategies are designated on the control plane, per managed name.

  In prior versions, if a domain was set up with the DNS64 flag, and DNS returned an IPv4 address within a range set up with NAT64 rules, Bowtie would return 0 records for A requests and return DNS64 answers for AAAA requests.

  This has been relaxed. Now, in the above scenario if the IPv4 address is routable, the A record response is not squashed.

• When IPv4 Support is enabled on the controller for a given device, A records are no longer dropped by default. DNS Strategies are designated on the control plane, per managed name.

• Bowtie now refuses to install IPv4 routes that conflict with local system routes.

  For instance, if you have a Bowtie site with a 192.168.4.0/22 network and you use it on a computer that is connected to a 192.168.4.0/22 network, Bowtie will not install routes to the Bowtie 192.168.0.0/22 network to ensure that it does not break local connectivity.

  This also applies when the Bowtie network length is shorter than the local network. If the Bowtie site is 192.168.4.0/22 and the local network is 192.168.0.0/20 Bowtie will not install routes the Bowtie 192.168.0.0/22 to ensure that it does not break local connectivity.

  Operating systems preferentially route to the most specific network so if the Bowtie site has a network length longer than the local network, Bowtie routes are installed. For example, if the Bowtie site is 192.168.4.0/22 and the local network is 192.168.5.0/24, Bowtie will install routes for 192.168.4.0/22. The operating system will route 192.168.5.0/24 to the local network, but the rest of the Bowtie network will be accessible.

  Servers located on the Bowtie site network can be accessed via NAT64 if they are not accessible via IPv4.

Server

Fixes

• A write lock was erroneously held through a read operation which did not close the transaction. Additionally an error in rolling back that transaction causes an unrecoverable panic. In some cases this cascades through the cluster. The offending operation has been repaired.

• Fixed an issue that caused updated control plane web assets to be incorrectly cached, causing inconsistent behavior after updating Controllers.

Features

• Controller images are now available in AWS GovCloud. See the Bowtie downloads page for AWS for a list of regions including those hosted in GovCloud.

• The DNS configuration page can now be sorted and filtered.

• Beta policy evaluation tool in the policy area. This page is for debugging and understanding the policy engine. It will show you the current state of the policy engine, and allow you to see how it is evaluating a given request.

Enhancements

• Updated the base network Controller appliance operating system to reflect the latest upstream package updates. This includes patch version updates to Grafana as well as patches to recent CVEs in packages like glibc.

  This update includes an update from Linux kernel version 6.1.75 to 6.1.78. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel, but this step is not required.

Helm Chart

Fixes

• Fixed an issue that caused the Bowtie server container to fail to emit its correct version string.

Features

• The Helm chart now accepts a list of environment variables to set under the server.env key for arbitrary settings like changing RUST_LOG to debug with a values file like the following:

  server:
    env:
    - name: RUST_LOG
      value: debug
  

24.02.003

Released on 22 February, 2024.

Server

Fixes

• Fixes an issue with policy order application.

Enhancements

• In addition to native cloud images for platforms like AWS, GCP, and Azure, raw disk images for each platform are now available for use in special cases such as import into currently-unsupported regions. Consult the platform-specific documentation under the Controller installation section for more information.

• Controller appliances for GCP are now available as native GCE images (in addition to raw disk image files). Note that public custom images are not listed over the GCP API and must be explicitly referenced by their resource identifiers when creating new GCE instances. Please refer to the GCP setup documentation and the GCP downloads page for additional information.

Meta Control Plane

Features

• Downloads now include buttons to copy curl or wget commands to the clipboard to aid with downloading packages in command line environments.

24.02.002

Released on 16 February, 2024.

Server

Fixes

• Device assignment and web application flows share some authentication code. All prior versions of Bowtie were failing to expire a client side cookie for device-id which made multiple tasks (web admin + device authentication) in the same browser behave incorrectly. This has been patched to expire the device-id on any successful authentication of the /password-login or SSO.

• Previously policy order rendering silently relied on the ID of the policies as well as the “order” field. Now ordering in the policy engine and the UI is consistent with the order field.

Features

• Added the ability to see user status and manually activate/deactivate users from the Web UI. This feature was previously exposed to the API only.

Client

Features

• The Help page has gained more fine grained status information.

24.02.001

Important

This release includes important changes that administrators should review.

Released on 14 February, 2024.

Server

Fixes

• Fixed some additional cases in which the bowtie-server process would crash when restarting, causing server daemon instability.

Features

• Operators may opt-in to pre-release builds of Controllers. To do so, either pass the --prerelease flag to the update command-line utility or set the BOWTIE_PRERELEASE environment variable if updating Controllers via automatic updates (for example, by including the line BOWTIE_PRERELEASE=1 in /etc/default/update).

• Bowtie now provides Controller network appliance images for Google Compute Engine (GCE) with EFI support.

• “Last Seen Version” is displayed on the Devices page of the web GUI.

Enhancements

• The foundational operating system for Controllers has been updated from NixOS 23.05 to 23.11.

  This update includes the following noteworthy version changes. If you rely on any of these packages for downstream integrations (such as observability metrics from opentelemetry-collector-contrib), ensure that the updated versions are compatible with your existing configuration (and update those configurations if necessary).

  Package	Old Version	New Version
  Linux kernel	6.1.38	6.1.75
  cloud-init	23.1.2	23.3.3
  grafana	9.x	10.x
  jq	1.6	1.7
  loki	2.8.6	2.9.4
  openssh	9.3p2	9.5p1
  opentelemetry-collector-contrib	0.77.0	0.87.0
  prometheus	2.44.0	2.49.0
  prometheus node_exporter	1.5.0	1.7.0
  promtail	2.8.6	2.9.4
  python	3.10.13	3.11.6
  tempo	2.1.1	2.3.0

Client

Features

• Bowtie on MacOS has gained the ability to operate on a port other than the standard DNS port of 53. To change ports, set the local_dns_listen_port option to an otherwise unused port in the configuration file in /etc/bowtie/configuration/.

  When this option is set to a value different than 53, Bowtie installs some pf firewall rules to redirect DNS queries from port 53 to the new port.

  Some of the applications that can claim port 53 and cause Bowtie to fail are Cirrus Labs’ Tart VM, old versions of Docker and the Internet Sharing facility of MacOS.

• On MacOS, Bowtie has gained the ability to auto-select the DNS port. If the configuration parameter local_dns_listen_port is set to 0, the port will be auto-selected. Port 53 will be chosen if it is available, otherwise a random port will be chosen. If a port other than 53 is selected manually or automatically, the MacOS firewall (pf) will be used to direct DNS traffic from port 53 to the chosen port.

  The default for local_dns_listen_port has been changed to 0. Set local_dns_listen_port to 53 to fail instead of using a firewall redirect if port 53 is unavailable.

Fixes

• The Default MTU for the VPN tunnel has been changed to 1280 on MacOS, matching the previous behavior on Linux.

24.01.004

Released on 1 February, 2024.

Server

Features

• Resource groups can now be deleted from the UI instead of just the API. Resource groups still in-use by policies are protected in the API.

24.01.003

Released on 31 January, 2024.

There are no release notes for this version.

24.01.002

Released on 30 January, 2024.

Client

Fixes

• MacOS: fix for log directory permissions issue that prevented tray app from starting for some

• Updated the Bowtie client UI framework that includes fixes for crashes that occurred on MacOS x86 platforms.

Server

Features

• Controller images compatible with ESXi VMWare hypervisor environments are now available as .ova downloads.

24.01.001

Released on 25 January, 2024.

Server

Features

• Controller events like updates are now recorded and displayed under the System Messages page of the control plane interface. Release notes are also provided on this page to quickly summarize changes. Consult the user documentation for additional information.

• Added ability to force device re-authentication after N number of minutes. Off by default. Go to the /settings page or update the org configuration API to activate.

• Individual resources may now be deleted when managing policies in the Control Plane web interface.

Client

Features

• Linux clients have gained the option dns_supervisor, (--dns-supervisor on the command line). The default value is Systemd, the old behavior may be obtained through the value BowtieService.

  The main functional difference between these two options is that the bowtie-dns logs are separated into the bowtie-dns journal with the Systemd option, rather than appearing in the bowtie-service journal.

  Behind the scenes the Systemd option creates a bowtie-dns systemd service, and bowtie-service starts and stops this new service when appropriate. With BowtieService as the option, bowtie-dns is forked from bowtie-service.

• The OS-specific Wireguard providers have been split into two separate Wireguard and routing providers. This is provided by the new command line option --routing-provider (routing_provider in configuration files) in addition to the previous option --wireguard-provider (wireguard_provider).

• bowtie-service has gained the option --dry-run (dry_run = true in configuration files). If set, bowtie-service will log the commands it would at the INFO level rather than actually run them. See –verbose/–silent for more information on log levels.

  This is only effective for providers that use command line commands. On MacOS, the default boringtun provider does not use command line commands. --dry-run should be coupled with --wireguard-provider=wireguard-go or --wireguard-provider=boringtun-cli.

• A new button was added to the “Help” screen in the Bowtie tray application to trigger collection of Bowtie logs and diagnostic information. This creates a zip or tar file that can be then sent in to assist support.

  On MacOS or Linux, this diagnostic file may be generated manually by running bowtie-ctl inspect.

Deprecations

• On MacOS the option --wireguard-provider=boringtun-hybrid (wireguard_provider="BoringtunHybrid") is now gone. Use --wireguard-provider=boringtun (wireguard_provider="Boringtun") and --routing-provider=route-cli (routing_provider="RouteCli") instead. These are the defaults and recommended, so alternatively the CLI arguments or configuration file options may be removed. All the other wireguard providers (wireguard-go, boringtun, boringtun-cli) are still available and now gain the ability to specify a different routing provider.

  On MacOS the alternative routing provider is --routing-provider=route-socket (routing_provider=RouteSocket). This provider currently is in an alpha state, has known issues and is not recommended.

• On Windows, the option --local-dns-provider (aka local_dns_provider in configuration files), the default value net-powershell (aka NetPowershell) has been renamed to registry-injector (aka RegistryInjector).

Documentation

Features

• JSON-formatted changelogs are now available on the user documentation site under the /changelog.json URL.

• Release notes and changelogs are now available as feeds from the user documentation site. See the documentation for additional information.

23.12.003

Released on 13 December, 2023.

There are no release notes for this version.

23.12.002

Released on 13 December, 2023.

Server

Fixes

• Fixes a bug in DNS Block Lists where tasks were not automatically created when sending a URL and also an empty contents block.

23.12.001

Released on 6 December, 2023.

Server

Fixes

• Several improvements to DNS block lists:

  • Remote upstream URLs are now retrieved asynchronously in the background on update. This should result in much faster changes when updating DNS block lists.

  • DNS block list update tasks will remove themselves when their accompanying DNS block list resources are deleted.

  • DNS block list meta update tasks will remove themselves when no DNS block lists are defined and the meta update task is no longer necessary.

• Fixed an issue in which the bowtie-server service failed to log a correct version string when starting.

• Fixed an issue in which a call to update a Controller (via either interactive shell or automatic timer) could hang indefinitely when cloud-init fails to complete.

Features

• Added a flag to optionally disable the Secure option when sending Set-Cookie headers in development. This flag should not be set in production environments. Consult bowtie-server --help for usage.

Enhancements

• Updates the base operating system packages to reflect the latest upstream changes. This includes non-breaking changes for security fixes in packages for Loki and Promtail and openssl 3.

  This update includes an update from Linux kernel version 6.1.38 to 6.1.62. System services should continue to operate normally across kernel updates, but if you require that the system run on the newer kernel, you should follow-up with any update actions with a system reboot to run on the newer kernel.

Client

Features

• On MacOS, logs are now sent to the system’s unified log service rather than being stored in files.

  Logs may be accessed by using bowtie-ctl inspect, by using the MacOS console app or by using the log command:

  log stream --info --debug --predicate 'subsystem = "works.bowtie.bowtie-service"'
  

  A subsystem predicate of works.bowtie.bowtie will dump the logs from the tray application rather than the service.

• Added support for connecting to multiple organizations simultaneously.

  To use this feature, place one or more configuration files with an entrypoint line in the Bowtie configuration directory (on MacOS and Linux /etc/bowtie/configuration, and on Windows, C:\ProgramData\bowtie\configuration).

  For example, the following snippet shows a client configuration on Linux that will connect to two organizations with two separate controllers:

  $ cat /etc/bowtie/configuration/first.conf
  entrypoint = [ "https://controller.example.com" ]
  $ cat /etc/bowtie/configuration/second.conf
  entrypoint = [ "https://controller.rock.associates" ]
  

  Bowtie currently only supports DNS on a single organization at a time. To switch to a different organization, access the “pause” capability by right-clicking the Bowtie system tray icon.

• Added the ability to pause and resume a Bowtie connection. This may be accessed by right clicking on the Bowtie tray application and selecting “pause” or “resume”. These menu items are only respectively available if the connection is in the “Connected” or “Paused” state.

Fixes

• Fixed an issue which prevented the tray application from running for multiple users simultaneously.

Enhancements

• The .msi file now includes required dependencies for Windows 10 and above. The use of the exe is not recommended for most cases.

Documentation

Enhancements

• The client setup and usage documentation has been overhauled to provide complete instructions for installation, uninstall steps, and desktop application use.

• The control plane documentation has been expanded to cover all of the administrative web pages available on Controllers.

• Updated the Terraform knowledge base section to include examples illustrating how to provision bowtie_dns_block_list resources.

• Added screenshots and examples for client tray menus that include multiple organizations and pause/resume buttons.

• Updated the section about Terraform provider ordering to reflect updated capabilities of the native provider to authenticate lazily.

23.11.003

Released on 15 November, 2023.

Client

Features

• Names may be queried with ip- or ip. notation to get their Bowtie equivalent addresses and routes.

  ❯ dig AAAA ip.172.16.1.1
  ; <<>> DiG 9.18.18-0ubuntu0.22.04.1-Ubuntu <<>> AAAA ip.172.16.1.1
  ;; global options: +cmd
  ;; Got answer:
  ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 31800
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1
  
  ;; OPT PSEUDOSECTION:
  ; EDNS: version: 0, flags:; udp: 65494
  ;; QUESTION SECTION:
  ;ip.172.16.1.1.         IN  AAAA
  
  ;; ANSWER SECTION:
  ip.172.16.1.1.      60  IN  AAAA    fdc3:cbe8:ded7:7ad0:0:64:ac10:101
  
  ;; Query time: 4 msec
  ;; SERVER: 127.0.0.53#53(127.0.0.53) (UDP)
  ;; WHEN: Thu Nov 09 17:18:42 PST 2023
  ;; MSG SIZE  rcvd: 70
  

Server

Features

• policy objects now allow an optional order attribute to be added and the UI supports drag and drop ordering. order is not absolute and duplicates are allowed in the API. The web interface will attempt to rectify ordering into a linear fashion if given the opportunity. In order to preserve current behavior, existing ordering is matched upon upgrading to this version.

Fixes

• Updated the open-vm-tools package on VMWare Controller images to patch for the following CVEs:

  • CVE-2023-34058

  • CVE-2023-34058

23.11.002

Released on 6 November, 2023.

Server

Features

• Hosts can now be blocked from the client en-masse via DNS block lists. Block lists can be manually curated or regularly pulled from a plain text file over http or https.

• Adds the ability to set configuration options on the control plane.

  GET and POST against /-net/api/v0/organization/config can influence control plane operations such as the following:

  • If a user successfully authenticates after installing a Bowtie client, automatically approve the device.

    allow_device_approval_on_user_auth: bool
    

  • If a controller has a PSK, automatically approve it’s joining the control plane.

    allow_controller_approval_with_psk_only: bool
    

  These new options are configurable over the web interface under the “Settings” section in the navigation bar.

• If groups are passed in from SSO via the groups attribute, they will be parsed and applied to the user.

• The cloud-init executable is now included in the global system $PATH for any images that support cloud-init.

Documentation

Enhancements

• The documentation outlining how to use cloud-init and leveraging it for automated setup has been overhauled to include additional documentation for the skip-gui-init file and PSK parameter.

• Added some important information about how Bowtie Controllers automatically provision SSL/TLS with ACME providers. If you leverage ACME heavily in your organization and may be subject to rate limits or make use of alternative validation schemes like tls-alpn-01, you may want to consult the updated section on ACME caveats.

Features

• Added documentation outlining how to use the Bowtie Terraform provider.

23.11.001

Released on 1 November, 2023.

Server

Fixes

• Fixed an issue in which cloud-init hostnames were not properly ingested by some system services which could impact web endpoints functioning correctly.

23.10.012

Released on 31 October, 2023.

Server

Fixes

• Fixed an issue where a joining controller does not fully clean up before initializing in the cluster and becomes unbootable without manual intervention

Features

• Added the ability to initialize the first users from cloud init.

  If you create a file in cloud init at /var/lib/bowtie/init-users with email addresses and argon2 hashed passwords, those users will be granted administrative rights at first boot.

  To generate the hash for password “hunter 2” you would use this command:

  echo -n "hunter2" | argon2 $(uuidgen) -i -t 3 -p 1 -m 12 -e
  

  And then put the contents into this file

  issac@example.com:$argon2i$v=19$m=4096,t=3,p=1$ZDhmOTB..........
  

Documentation

Features

• Added an example of how to reset Caddy to stock defaults in the knowledge base documentation.

23.10.011

Released on 30 October, 2023.

There are no release notes for this version.

23.10.010

Released on 30 October, 2023.

Helm Chart

Fixes

• Fixed an issue in which pre-release Helm charts referenced an incorrect image name.

Features

• The image names for both the server and frontend containers is now exposed as a top-level values parameter.

  Their defaults remain the same:

  server:
    image: bowtie-server
  frontend:
    image: bowtie-assets
  

23.10.009

Released on 25 October, 2023.

Server

Fixes

• Fixed an issue in which the bowtie-server address binding might have caused traffic sent to containers to be dropped. This option is now configurable and has been fixed in the Bowtie Helm chart.

Features

• Added the ability to use API keys to manage device acceptance.

  To Create an API key you must be an Owner or Full Administrator User.

  Create a new API Key in the GUI or at /-net/api/v0/api_keys

  Keep the key and ID.

  ID: d973d102-8ae5-4c5f-b393-4302abbc5024
  KEY: bts1_b6a91c33-08a3-4069-a69c-c86feda404b1
  

  Now you can use it to get devices or set their allowed state

  curl -H 'Authorization: Basic ZDk3M2QxMDItOGFlNS00YzVmLWIzOTMtNDMwMmFiYmM1MDI0OmJ0czFfYjZhOTFjMzMtMDhhMy00MDY5LWE2OWMtYzg2ZmVkYTQwNGIx' https://controller.example.com/-net/api/v0/device
  

  Or More simply

  curl -u d973d102-8ae5-4c5f-b393-4302abbc5024:bts1_b6a91c33-08a3-4069-a69c-c86feda404b1 https://controller.example.com/-net/api/v0/device
  
  curl -u d973d102-8ae5-4c5f-b393-4302abbc5024:bts1_b6a91c33-08a3-4069-a69c-c86feda404b1 \
      -H "Content-Type: application/json"  \
      -d "{"devices": [{"id": "fd2d3981-d82e-406b-a953-35e9ce6dd39b", "state": "rejected"}]}" \
      https://controller.example.com/-net/api/v0/device/state \
  

Documentation

Features

• Added release notes for software releases, providing additional context for updates. These are visible on the release notes documentation page.

Meta Control Plane

Fixes

• Fixed an issue with honoring per-user key tokens when using the /latest/ API endpoints.

23.10.008

Released on 20 October, 2023.

Server

Fixes

• The bowtie-server service now restarts in all cases, not just when exiting with a non-zero status code.

23.10.007

Released on 20 October, 2023.

Server

Features

• A new flag called BOWTIE_ORG_RESPONSE_BEHAVIOR has been added that controls how the Controller API will respond to API calls against the /organization endpoint. In a future release, the /organization endpoint will only return content for accepted devices, and setting BOWTIE_ORG_RESPONSE_BEHAVIOR=accepted-devices-only will preview this behavior that will eventually become the default mode of operation. Existing installations will retain the current behavior – this flag is meant as an aid in preparing operators for future changes to Controller defaults.

  To prepare for the new future default behavior, operators should set this variable for existing Controllers in preparation for the default to change. For example, on existing Controllers or as a bootstrapping step in cloud-init user data, creating a file like /etc/bowtie-server.d/org-response.conf that includes BOWTIE_ORG_RESPONSE_BEHAVIOR=accepted-devices-only will enable the behavior.

23.10.006

Released on 18 October, 2023.

There are no release notes for this version.

23.10.005

Released on 18 October, 2023.

Server

Fixes

• Some Grafana authentication settings have been made more secure by default.

23.10.004

Released on 17 October, 2023.

Server

Features

• Added a new Prometheus exporter for use in observability collection and dashboards for wireguard. The following screenshot demonstrates the type of data available for use, including wireguard interface network traffic and last-seen timestamps for wireguard peers.

  

  Default wireguard dashboards in Grafana

23.10.003

Released on 16 October, 2023.

There are no release notes for this version.

23.10.002

Released on 6 October, 2023.

There are no release notes for this version.

23.10.001

Released on 6 October, 2023.

There are no release notes for this version.

23.09.001

Released on 19 September, 2023.

There are no release notes for this version.

23.08.021

Released on 30 August, 2023.

There are no release notes for this version.

23.08.020

Released on 28 August, 2023.

There are no release notes for this version.

23.08.019

Released on 26 August, 2023.

There are no release notes for this version.

23.08.018

Released on 25 August, 2023.

There are no release notes for this version.

23.08.017

Released on 25 August, 2023.

There are no release notes for this version.

23.08.016

Released on 25 August, 2023.

There are no release notes for this version.

23.08.015

Released on 25 August, 2023.

There are no release notes for this version.

23.08.014

Released on 24 August, 2023.

There are no release notes for this version.

23.08.013

Released on 21 August, 2023.

There are no release notes for this version.

23.08.012

Released on 17 August, 2023.

There are no release notes for this version.

23.08.011

Released on 16 August, 2023.

There are no release notes for this version.

23.08.010

Released on 15 August, 2023.

There are no release notes for this version.

23.08.009

Released on 14 August, 2023.

There are no release notes for this version.

23.08.008

Released on 11 August, 2023.

There are no release notes for this version.

23.08.007

Released on 10 August, 2023.

There are no release notes for this version.

23.08.006

Released on 9 August, 2023.

There are no release notes for this version.

23.08.005

Released on 8 August, 2023.

There are no release notes for this version.

23.08.004

Released on 4 August, 2023.

There are no release notes for this version.

23.08.003

Released on 4 August, 2023.

There are no release notes for this version.

23.08.002

Released on 3 August, 2023.

There are no release notes for this version.

23.08.001

Released on 3 August, 2023.

There are no release notes for this version.

23.07.008

Released on 31 July, 2023.

There are no release notes for this version.

23.07.006

Released on 31 July, 2023.

There are no release notes for this version.

23.07.007

Released on 25 July, 2023.

There are no release notes for this version.

23.07.005

Released on 25 July, 2023.

There are no release notes for this version.

23.07.004

Released on 20 July, 2023.

There are no release notes for this version.

23.07.003

Released on 12 July, 2023.

There are no release notes for this version.

23.07.002

Released on 11 July, 2023.

There are no release notes for this version.

23.06.009

Released on 29 June, 2023.

There are no release notes for this version.

23.06.008

Released on 28 June, 2023.

There are no release notes for this version.

23.06.007

Released on 28 June, 2023.

There are no release notes for this version.

23.06.006

Released on 26 June, 2023.

There are no release notes for this version.

23.06.005

Released on 25 June, 2023.

There are no release notes for this version.

23.06.004

Released on 25 June, 2023.

There are no release notes for this version.

23.06.003

Released on 22 June, 2023.

There are no release notes for this version.

23.06.002

Released on 21 June, 2023.

There are no release notes for this version.

23.06.001

Released on 15 June, 2023.

There are no release notes for this version.