What are Cloud costs overruns?¶
Infrastructure costs that communities incur outside of an active service agreements
Costs incurred by 2i2c because invoices cannot (or have not) been sent to communities
In general, any cloud costs that 2i2c can’t recoup from a community
Cloud costs monitoring process¶
2i2c’s cloud costs monitoring process consists of two phases.
Phase 1: Monitoring and reducing cost runaways¶
Preventing cost runaway involves actively responding to cloud cost alerts and escalated community concerns (from Freshdesk).
Cloud cost alerts¶
The P&S team configures public clouds (e.g. AWS, Azure) systems to raise usage alarms at specific thresholds.
When cloud cost alerts are triggered, these are the steps that should taken by the P&S team:
When a cloud account is set up, we set a threshold for sending alerts
If an account’s forecast (not actual) cost goes above the threshold, an alert email is sent to support@2i2c.org
The P&S support triager reviews alert emails
Excessive spikes (typically >20%, open to the triager’s judgement) will trigger a formal investigation by a member of the P&S team and where necessary a report is sent to the impacted community
In the event of a cost overrun, the Delivery manager will decide if 2i2c will consume the cost and not pass it down to the community
DE will communicate to CS&S if a discount should be applied to the communities next invoice
Escalated community concerns¶
Community escalated cost concerns should be handled in a similar manner to cloud cost alerts.
A message is sent to FreshDesk (by a concerned community)
P&S support triagers trigger a discussion within P&S (and BD where necessary)
The P&S team will investigate the cost concern and advise the DE team of possible next steps. This should include the following information:
Is this a legitimate cost overrun
Is 2i2c at fault
Does P&S believe that a discount should be issued
In the event of a cost overrun, the Delivery manager will decide if 2i2c will consume the cost and not pass it down to the community
In the event of a cost overrun, P&S support triagers will communicate costs/discount implications (and root causes) to impacted communities
DE will communicate to CS&S if a discount should be applied to the communities next invoice
Phase 2: Invoicing communities¶
Reporting and invoicing consist of getting the cloud costs from our different platforms, assessing for anomalies and dispatching adjusted costs for invoicing.
This phase involves the following steps:
Gather costs: Cloud costs are gathered by the P&S team on a monthly basis.
Review costs: These costs are communicated to DE for review via the #billing slack channel. DE is responsible for:
Flagging anomalies to P&S
Triggering cost anomaly investigations
Coordinating discounts with CS&S
Dispatch for invoicing: Cloud costs are sent to CS&S for invoicing. 4. This is done by consolidating the CSV files from P&S to a google cloud sheet 5. P&S is notified on Asana that the cloud costs are ready for invoicing
See How to invoice communities for cloud costs for the invoicing process and reference documents.
Decommissioning hubs¶
In some extreme cases, 2i2c will have to discontinue services to communities.
The process that governs these decisions is below:
The BD team (and representatives from DB and P&S) will review which services agreements have lapsed and are still consuming cloud services in the Biz Dev Sales Sync meeting
DE and P&S team members will review Engagements in the P&S Project management meeting, identifying hubs that are candidates for decommission based
They will then notify the BD team via the Biz Dev Sales sync meeting
The BD team is responsible for extending the hub decommissioning date (by 2-week increments) to facilitate renewal conversations in those instances where a service agreement has lapsed.
If communities have lapsed (or missing) services agreements for more than 6-weeks and are still receiving cloud services for that period of time, the BDL will make a decision if their cloud services should be terminated.