Pragma Engine 0.0.101

May 2024

What’s New? #

The Bans feature has new capabilities for specifying time duration, adding ban reasons, commenting, and auditing. #

Using the updated Bans feature, you can now set a specific time duration and provide a reason for each ban. Customize ban reasons to align with your game’s requirements, covering anything from terms of service violations to cheating. Internal comments can be included to offer further context on the ban. In addition, you can track who added or revoked each ban.

Observability tools for platform monitoring and issue identification. #

With the latest update to Pragma’s observability suite, we introduce new metrics and Grafana dashboards designed to enhance your ability to monitor platform health and respond to issues efficiently. Our enhanced overview dashboards allow for quick assessments of platform status, making it easy to spot problems at a glance. Detailed dashboards provide deep dives into specific areas of the platform, enabling precise diagnostics and troubleshooting. The addition of preconfigured alerts for production environments means that signs of platform instability immediately notify Pragma on-call engineers, reducing the time to resolution and minimizing disruption to your operations.

Network and database flexibility for managed infrastructure. #

Pragma’s managed infrastructure has been updated to now support the use of port 443, ensuring all players can reliably connect to your platform, even under restrictive network conditions. Additionally, our managed infrastructure can more easily stand up multiple databases per shard, allowing for tailored horizontal and vertical scaling adjustments.

Contents #

Summaries | Summarized release notes.

Full Notes | Full notes including descriptions and additional details.

Summaries #

Features #

• [Multiplayer] SDK Party state maintained during unintentional disconnects from Pragma. | full note
• [Multiplayer] New Load Testing Game Server service allows you to run load tests both locally and inside Pragma. | full note
• [Multiplayer] New initialization pattern in Unreal Party SDK communicates whether the player is in a party. | full note
• [Multiplayer] New Matchmaking getQueueName() function allows you to customize how queues appear in metrics and logging. | full note
• [Multiplayer] Match Capacity service was fully removed in favor of the Fleet service. full note
• [Accounts] Retrieve social and player identities by using GetSocialIdentitiesV1, GetSocialIdentitiesByProviderAccountIdsV1, and GetPlayerIdentitiesV1. | full note
• [Accounts] Get a list of active bans on an account in the Account Plugin using PragmaAccount.getActiveBans. | full note
• [Accounts] Specify a ban duration, reason, and comment using the new fields. | full note
• [Accounts, Portal] Specify if a ban is for all game shards or a specific game shard. | full note
• [Accounts, Portal] Add a ban duration, reason, and comment through the Social Operator Portal. | full note
• [Engine] Updated PragmaResult for easier extendable responses and with additional helper functions for more readable and clean code. | full note
• [Engine, Portal] Enabled xForwarded headers for Operator and Player gateways. | full note
• [Infra & Tooling] Updated Multiplayer and Matchmaking Grafana dashboards to show all Pragma-authored metrics. | full note
• [Infra & Tooling] New Node Count panel in Grafana for real-time node monitoring. | full note
• [Infra & Tooling] MySQL slow query logs for insights into opportunities for database query optimization. | full note
• [Infra & Tooling] Use content auto-deploys to minimize manual deployments. | full note
• [Infra & Tooling] Configure your platform to use port 443 to avoid issues with restrictive network policies. | full note
• [Infra & Tooling] Stand up multiple databases for your environment so that you can horizontally and vertically scale your platform. | full note
• [Portal] If you want to sort by display name, use the table sort controls. | full note

Deprecations #

• [Multiplayer] sdk | Several functions in the Unreal GameLoopApi have been deprecated, as their functionality is being relocated within the API. | full note
• [Friends] sdk | The Friend API ForceReset() function has been deprecated in favor of ForceSync(). | full note
• [SDK] sdk | TFuture functions have been deprecated in the Unreal SDK. | full note

Integrations #

• [Multiplayer] config | Discontinue use of the Match Capacity service and adjust Fleet service settings. | full note
• [Multiplayer] sdk | Change Unreal SDK FPragmaFriend and FriendOverview functions to use TOptional return types. | full note
• [Multiplayer] sdk | Update Unreal SDK to use the GameLoopApi.Initialize() function. | full note
• [Multiplayer] sdk | Use GameLoopApi Party events instead of Session service notifications to listen for Party ID changes. | full note
• [Accounts] platform | Add ExtBanReason to the accountRpcExt proto. | full note
• [Accounts] sdk, platform, portal | If you’re using the Bans feature, you’ll need to update your code to account for the new bans response structure. | full note
• [Accounts] platform | If you use PragmaAccount.getBanHistory, you’ll need to make modifications to accept the new response structure. | full note
• [Accounts] platform | Update references of GetBansV1Response::banRecords. | full note
• [Accounts] sdk | Update Unreal Player SDK GetBans delegate. | full note
• [Accounts] platform | If you use the BanAccount RPC, you’ll need to add banScope. | full note
• [Accounts] sdk | banReason and comment are now required parameters. | full note
• [Accounts] platform, sdk | Update references of GetPragmaPlayerIdsForProviderIdsV2Request to GetPlayerIdentitiesByProviderAccountIdsV1Request. | full note
• [Engine] platform | Update any uses of PragmaResultResponse and PragmaResultError. | full note

Fixes & Improvements #

• The Unreal SDK OnPrivatePlayerChanged event now broadcasts dependably when the player’s ExtPrivatePlayer payload has changed. Previously, an oversight caused this event not to be checked and fired when party updates occurred.

• The GetPragmaPlayerIdsForProviderIdsV2Request endpoint (which has now been renamed to GetPlayerIdentitiesByProviderAccountIdsV1Request) no longer throws an error when no results are found. Previously, the endpoint would throw an error if no player identities were found for the given provider account IDs. The endpoint now returns an empty list of player identities if none are found.

• Optimized the time it takes to load the account overview and player support pages.

• Fixed a bug for OAuth redirects when using a load balancer or reverse proxy.

• Pragma-managed infrastructure supports writing metrics to multiple Prometheus endpoints, now both for AWS and non-AWS-managed Prometheus.

Docs #

• [New Tutorial] Added new Account Plugin tutorial to demonstrate two different scenarios for updating a player’s Pragma Account display name.
• [Updated Concepts] Updated Player Data Overview pages with links to relevant implementation tasks and reference details.

Full Notes #

Features #

[Multiplayer] SDK Party state maintained during unintentional disconnects from Pragma. #

Description: Previously, on any disconnect, the party state in the SDK would be wiped. In certain cases, the platform could still think the player was in the party, leading to party state conflicts. With version 0.0.101, the SDK party state is maintained in the case of unintentional disconnect.

Details: The SDK party state will be wiped when calling LogOut on the Player API.

[Multiplayer] New Load Testing Game Server service allows you to run load tests both locally and inside Pragma. #

Description: The new Load Testing Game Server service can act as multiple game servers, which is useful for running load tests both locally and inside Pragma.

Details: To add the new service to your project:

• Run the following to copy the custom services into your 5-ext project: make add-load-test-services
• Update the serviceConfigs and pluginConfigs blocks in your config file as follows:

game:
  core:
  serviceConfigs:
    FleetServiceConfig:
      serverPoolManagementPolicies:
        1:
          id: "LOADTESTING_MANAGEMENT_POLICY"
          ...
    LoadTestingGameServerServiceConfig:
      capacityPerServer: 1
      partnerConnectionInfo:
        protocol: "http"
        host: "127.0.0.1"
        port: 10100
      clientInitialHeartbeatPeriodMillis: 1000
      matchLauncher: "demo.loadtesting.LocalMatchLauncher"
  pluginConfigs:
    FleetService.fleetPlugin:
      class: "demo.fleet.LoadTestingGameServerFleetPlugin"
      config:
        serverPoolId: "LOADTESTING"
        managementPolicyConfigId: "LOADTESTING_MANAGEMENT_POLICY"

• Optionally, delete the Skeleton Fleet Plugin, along with the following files from the 5-ext/ext/src/main/kotlin/demo/fleet folder:
  • BaseMatchLauncher.kt
  • GameFlowMatchLauncher.kt
  • LocalMatchLauncher.kt
  • SkeletonFleetPlugin.kt
  • SkeletonInMemoryGameServer.kt
  • SkeletonMatchLauncher.kt

[Multiplayer] New initialization pattern in Unreal Party SDK communicates whether the player is in a party. #

Description: Prior to this release, the GameLoopApi determined whether a player was in a party based on direct RPC interactions with the party system. This design did not provide a way to retrieve the latest information from the platform if it were to become out of sync. The GameLoopApi now requires calling an Initialize() function prior to use. This step will synchronize party states so that newly created SDK sessions can correctly determine what party a player is in, if any.

Details: When the OnComplete callback to Initialize() fires, calling the new GetPartyState() returns an IPragmaPartyState* object that can be used to determine the state of the user’s party. In addition, the new ForceSync() function forcibly triggers an update of the game client’s cached party state.

Related notes: See the Update Unreal SDK to use the GameLoopApi.Initialize() function integration note.

[Multiplayer] New Matchmaking getQueueName() function allows you to customize how queues appear in metrics and logging. #

Description: Previous to this release, queue names appeared in metric reports and logs as a string version of the queue’s ExtMatchmakingKey. To increase ease-of-use, you can now use the Matchmaking Plugin’s getQueueName() function to generate a concise name for the queue. This allows you to consolidate queue metrics based on shared ExtMatchmakingKey values.

Details:

fun getQueueName(extMatchmakingKey: ExtMatchmakingKey): String {
   return "undefinedQueueName"
}

The getQueueName() function will be called during instantiation of a new matchmaking queue or when a metric/log requires a queue name and doesn’t already have access to the queue.

[Accounts] Retrieve social and player identities by using GetSocialIdentitiesV1, GetSocialIdentitiesByProviderAccountIdsV1, and GetPlayerIdentitiesV1. #

Description: The following are new endpoints for retrieving social and player identities:

• GetSocialIdentitiesV1: retrieves a list of all social identities by social IDs
• GetSocialIdentitiesByProviderAccountIdsV1: retrieves a list of all social identities by provider account IDs
• GetPlayerIdentitiesV1: retrieves a list of all player identities by player IDs

Details: This also introduces a new account data type PublicSocialIdentity which is a version of SocialIdentity intended for public distribution.

[Accounts] Get a list of active bans on an account in the Account Plugin using PragmaAccount.getActiveBans. #

Description: You can now specify whether you want a list of active bans or all ban history.

Details: The following are two methods on the PragmaAccount to retrieve ban history depending on your needs:

• getActiveBans: returns a list of currently active bans
• getBanHistory: returns a list of all ban history

[Accounts] Specify a ban duration, reason, and comment using the new fields. #

Description: Operator, Partner, and Service sessions can now specify a duration, reason, and comment when banning an account.

Details: The new fields durationInMillis, banReason, and comment have been added to the following ban account endpoints: - AccountRpc.BanAccountOperatorV1Request - AccountRpc.BanAccountPartnerV1Request - AccountRpc.BanAccountServiceV1Request

The default value of 0 for durationInMillis will make the ban permanent.

Example:

{
    "requestId": 1,
    "type": "AccountRpc.BanAccountPartnerV1Request",
    "payload": {
        "pragmaSocialId": "077ef10f-af39-4d3c-b930-095c5735caa9",
        "banScope": "BAN_SCOPE_ALL_GAME_SHARDS",
        "durationInMillis": "86400000",
        "banReason": "EXT_BAN_REASON_CUSTOM",
        "comment": "Player violated terms of service.",
    }
}

[Accounts, Portal] Specify if a ban is for all game shards or a specific game shard. #

Description: You can use the new parameter banScope to specify if the account is receiving a ban for all game shards or a specific game shard.

Details: You can now specify what game shards an account is banned from in the Social Operator Portal. banScope has been added to the following RPC endpoints for adding a ban:

• BanAccountPartnerV1Request
• BanAccountServiceV1Request
• BanAccountOperatorV1Request

banScope has also been added to the Ban proto object that is returned from all get bans and add ban calls.

[Accounts, Portal] Add a ban duration, reason, and comment through the Social Operator Portal. #

Description: You can now add a ban duration, reason, and comment when banning players through the Social Operator Portal. The Ban table now has a “Time Remaining” column. You have the option to choose “Temporary” or “Permanent” for the ban. When “Temporary” is selected, you can select either from one of the preset durations or enter a custom duration. Ban reasons can be customized to suit your game’s requirements. Internal comments can be added for additional context on the ban.

Details: New swappable Portal components for duration, reason, and comment can be added to the Ban tab under each account page.

See the Bans concepts page for more information.

[Engine] Updated PragmaResult for easier extendable responses and with additional helper functions for more readable and clean code. #

Description: PragmaResult has new names for PragmaResultResponse and PragmaResultError to better indicate whether the result is a success or error. PragmaResult also has new helper functions for cleaner code logic.

Related notes: See the Update any uses of PragmaResultResponse and PragmaResultError integration note.

[Engine, Portal] Enabled xForwarded headers for Operator and Player gateways. #

Description: We’ve added the capability to enable XForwarded headers for both Operator and Player gateways. XForwarded headers are useful when these gateways are behind a load balancer or reverse proxy. The XForwarded headers will preserve the original request protocol and port, allowing us to properly handle OAuth redirects.

[Infra & Tooling] Updated Multiplayer and Matchmaking Grafana dashboards to show all Pragma-authored metrics. #

Description: We’ve updated the existing Multiplayer and Matchmaking Grafana dashboards within the local Docker development environment and in hosted Pragma deployments. These updated graphs include all Pragma-authored metrics related to multiplayer and matchmaking features.

[Infra & Tooling] New Node Count panel in Grafana for real-time node monitoring. #

Description: The Shard Overview Grafana dashboard now includes this new panel to display the count of Pragma nodes by cluster name. This update eliminates the need to manually check node status through platform commands by giving you an at-a-glance-view of node status across their topology. You can use this feature to make sure shards are operating as expected, as a deviation from expected numbers of running nodes is a strong indicator of platform instability.

[Infra & Tooling] MySQL slow query logs for insights into opportunities for database query optimization. #

Description: Pragma-managed infrastructure supports enabling MySQL slow query logs which provide visibility into queries that exceed a predetermined time threshold. This helps identify inefficient or poorly optimized queries that can be further optimized. By optimizing database resource utilization, you can reduce the load on your database servers, which can defer or eliminate the need to scale up hardware resources, reducing cost over time. You can read more about MySQL slow query logs on the AWS documentation site. This feature is supported in versions 0.0.95 and later.

To enable this feature, contact your customer support representative with the shard to configure and time threshold to use.

[Infra & Tooling] Use content auto-deploys to minimize manual deployments. #

Description: To facilitate a content deployment process that requires minimal manual intervention, you can request for any branch to have automatic content deploys to any shard with the additional option to do a deploy without a restart. This feature eliminates the need to manually initiate the deploy process through Buildkite and click through the deployment options. Instead, you can have a content-only commit automatically initiate deployment of those changes and eliminate platform downtime by deploying without a restart. This feature is supported in versions 0.0.95 and later.

To enable this feature, contact your customer support representative with the branch and shard you’d like associated.

[Infra & Tooling] Configure your platform to use port 443 to avoid issues with restrictive network policies. #

Description: This feature allows you to set up your platform to use the standard port 443 for game and social connections to avoid restrictive network policies. Some places, like university dorms, might block traffic on non-standard ports, such as Pragma’s default ports of 10000 for game traffic and 11000 for social traffic. To have confidence that all your players are able to connect to your platform, we recommend updating the game and social ports to use 443 in your engine settings and the Pragma SDK, and then inform Pragma about these changes so that the necessary infrastructure updates can occur in tandem.

For more information, contact your Pragma support representative for the full guide.

[Infra & Tooling] Stand up multiple databases for your environment so that you can horizontally and vertically scale your platform. #

Description: Pragma’s managed infrastructure now supports more easily standing up multiple databases for each shard so that you can scale your platforms horizontally. Previously constrained to vertically scaling a single database instance, you can now specify databases for each service. Additionally, you have the flexibility to specify the size and type of each database instance and its read-replicas, allowing for tailored scaling adjustments that are cost-efficient. This feature is supported in versions 0.0.95 and later.

To enable this feature, contact your customer support representative and provide the database identifier(s) you plan to use, specify the shard where you want the changes implemented, and mention any special requirements. Our team will coordinate with you to schedule the downtime required to update the infrastructure and deploy your platform.

[Portal] If you want to sort by display name, use the table sort controls. #

Description: Account entries used to be default ordered by display name. Now they are ordered by creation time. Use the table sort options at the top of the List Accounts table.

Deprecations #

[Multiplayer] sdk | Several functions in the Unreal GameLoopApi have been deprecated, as their functionality is being relocated within the API. #

Description: The following table lists the functions in the Unreal PragmaGameLoopApi class that have been deprecated, along with suggested replacements:

The deprecated functions are scheduled to be removed in Pragma Engine version 0.0.102.

[Friends] sdk | The Friend API ForceReset() function has been deprecated in favor of ForceSync(). #

Description: To keep our cache patterns consistent across the Unreal API layer, we’ve replaced the Friend API ForceReset() function with ForceSync(). The functionality remains the same. If your SDK code currently calls ForceReset(), it should be updated to instead call ForceSync().

The ForceReset() function is scheduled to be removed in Pragma Engine version 0.0.102.

[SDK] sdk | TFuture functions have been deprecated in the Unreal SDK. #

Description: Unreal SDK functions that return a TFuture value have been deprecated. If you use an Unreal function that returns a TFuture, instead use the analogous function that returns void and takes a callback.

The deprecated functions are scheduled to be removed in Pragma Engine version 0.0.102.

Integrations #

[Multiplayer] config | Discontinue use of the Match Capacity service and adjust Fleet service settings. #

Description: The Pragma Match Capacity service was deprecated in version 0.0.99 and fully removed with this release. To account for this, Pragma Fleet service users no longer need to set the useFleetService configuration value to enable the Fleet service.

Integration steps:

• config | In your configuration file, in the GameInstanceServiceConfig block, delete the useFleetService entry.
• If you have not already migrated to the Fleet service, ask your customer support representative for the Fleet Migration Guide for additional necessary integration steps.
• Upon upgrading to 0.0.101 please reach out to your customer support representative so that the Pragma team can remove the Match Capacity service from your topology.

[Multiplayer] sdk | Change Unreal SDK FPragmaFriend and FriendOverview functions to use TOptional return types. #

Description: To better represent the returned data, the return types of the following Unreal SDK functions were changed from a TSharedPtr to a TOptional:

• FPragmaFriend::PlayerId()
• FPragmaFriend::Presence()
• FPragmaFriendOverview::PlayerId()

Integration steps:

• sdk | Update use of the above functions to expect a TOptional return type.

[Multiplayer] sdk | Update Unreal SDK to use the GameLoopApi.Initialize() function. #

Description: The new GameLoopApi.Initialize() function initializes the client’s party API state and synchronizes it with the platform’s party state. Previously, the SDK would always reset its locally-cached party state on startup.

If the player is already in a party when they login (such as when a disconnect/reconnect occurs), calling initialize() will fire the GameLoopApi notifications related to joining a new party.

Integration steps:

• GameLoopApi.Initialize() must be called prior to using the GameLoopApi within the Pragma Unreal SDK. Initialization can only happen after login, and the API must be reinitialized after any subsequent logins.
• Locate SDK code that creates Pragma player sessions and invoke UPragmaGameLoopApi::Initialize(const FOnCompleteDelegate& OnComplete).
• After the OnComplete delegate within that function has called back, GameLoopApi will be properly initialized and Player->GameLoopApi()->GetPartyState() will return a non-null pointer IPragmaPartyState*.

Related notes: See New Initialization pattern in Unreal Party SDK feature note.

[Multiplayer] sdk | Bind GameLoopApi events on login #

Description: GameLoopApi event bindings are cleared on logout. Previously, the delegates were not cleared.

Integration steps:

• sdk | Endure GameLoopApi events are bound/rebound on login

[Multiplayer] sdk | Use GameLoopApi Party events instead of Session service notifications to listen for Party ID changes. #

Description: The Session service is no longer responsible for tracking the player client’s party ID attribute. Instead, the new Multiplayer service tracks the party ID that a player is associated with.

Integration steps:

• sdk | If you use the Session service OnGameSessionChangedV1Notification to listen for party ID change events, instead use the GameLoopApi OnLeftParty and OnJoinedParty events.

[Accounts] platform | Add ExtBanReason to the accountRpcExt proto. #

Description: There is a new enum ExtBanReason for supporting ban reasons.

Integration steps:

• platform | Copy the ExtBanReason enum from platform/1-protos/proto-exts/src/main/proto/shared/accountRpcExt.proto into platform/5-ext/ext-protos/src/main/proto/shared/accountRpcExt.proto.

[Accounts] sdk, platform, portal | If you’re using the Bans feature, you’ll need to update your code to account for the new bans response structure. #

Description: The return type of the GetBansBySocialId and BanAccount endpoint was changed to a BanRecord type containing a Ban type.

Integration steps:

• sdk | Update PartnerBansApi::BanAccountFromGameShard delegate function to accept the new response structure.
• platform | Update the following endpoints to accept the new response structure:
• GetBansBySocialIdServiceV1Response
• GetBansBySocialIdPartnerV1Response
• GetBansBySocialIdOperatorV1Response
• BanAccountServiceV1Response
• BanAccountPartnerV1Response
• portal | If you have a custom portlet that uses the response of the GetBansBySocialIdOperatorV1 request you will need to account for the new banRecord wrapper of the ban object.

[Accounts] platform | If you use PragmaAccount.getBanHistory, you’ll need to make modifications to accept the new response structure. #

Description: getBanHistory now returns a ban record instead of just a ban. You’ll need to modify your code to take in the new ban history response structure.

Integration steps:

• platform | Update your code to accept the new response structure.

[Accounts] platform | Update references of GetBansV1Response::banRecords. #

Description: The return type of the GetBansBySocialId and BanAccount endpoint was changed to a BanRecord object which contains a Ban object.

Integration steps:

• platform | Update references of GetBansV1Response::banRecords to GetBansV1Response::bans.

[Accounts] sdk | Update Unreal Player SDK GetBans delegate. #

Description: The Unreal Player SDK GetBans delegate function was updated with the FPragmaBan struct. The Player GetBans delegate was being passed the raw FPragma_Account_GetBansV1Response instead of a TArray of FPragmaBan objects. This was changed to bring it inline with the OnAccountBanned notification type.

Integration steps:

• sdk | If you are using the Player Bans API GetBans function, you will need to change the delegate type.

[Accounts] platform | If you use the BanAccount RPC, you’ll need to add banScope. #

Description: banScope must now be specified on the following RPC requests to ban an account:

• BanAccountPartnerV1Request
• BanAccountServiceV1Request
• BanAccountOperatorV1Request

Integration steps:

• platform | Add banScope when using an RPC endpoint for adding a ban.

[Accounts] sdk | banReason and comment are now required parameters. #

Description: banReason and comment are now required parameters in BanAccountFromGameShard.

Integration steps:

• sdk | banReason and comment have to be provided in the constructor of the method.

[Accounts] platform, sdk | Update references of GetPragmaPlayerIdsForProviderIdsV2Request to GetPlayerIdentitiesByProviderAccountIdsV1Request. #

Description: GetPragmaPlayerIdsForProviderIdsV2Request has been renamed for a more accurate description of the endpoint.

Integration steps:

• platform, sdk | Update references of GetPragmaPlayerIdsforProviderIdsV2Request to GetPlayerIdentitiesByProviderAccountsIdsV1Request.

[Engine] platform | Update any uses of PragmaResultResponse and PragmaResultError. #

Description: PragmaResultResponse and PragmaResultError have been replaced with PragmaResult.Success and PragmaResult.Failure. PragmaResult also has new helper functions available to replace usages of a when case. Contact customer support for more help and details.

Integration steps:

• platform | Replace PragmaResultResponse and PragmaResultError with the new types:

• platform | We recommend updating use cases of PragmaResult to use the new helper functions instead of a when case. Below is an example that uses the new onFailure().

  • Before:

    when (service.requestRpc(request, ArbiterLeaderboardRpc.AddScoreServiceV1Response::class)) {
    is PragmaResultResponse -> {}
        is PragmaResultError -> {
            logger.error("Failure to add leaderboard score: ${leaderboardInfo.leaderboardId.toUUID()} ${leaderboardInfo.identifier} ${leaderboardInfo.score}")
        }
    }
    

  • After:

    service.requestRpc(request, ArbiterLeaderboardRpc.AddScoreServiceV1Response::class).onFailure {
    logger.error("Failure to add leaderboard score: ${leaderboardInfo.leaderboardId.toUUID()} ${leaderboardInfo.identifier} ${leaderboardInfo.score}")
    }
    

• platform | Update the load test modules generated from 5-ext module templates with the new PragmaResult types.

  • If you haven’t made any modifications to the load testing modules, delete the generated load-test and load-test-core modules under 5-ext and run make ext to regenerate the modules.
  • If you have made any modifications to the load-test module, manually update this module.
    • If you have also made modifications to load-test-core, manually update this module as well.
    • If you have not made modifications to load-test-core, delete load-test-core and run make skip-tests ext to regenerate the module with the new PragamResult type changes.