What makes up cloud costs and how to control them

We pass through cloud costs directly to our communities in a transparent manner. This encourages us to continually reduce the cloud costs for our communities, and helps them understand how their decisions affect their cloud bill. See Who owns the cloud account for whether you or 2i2c should own the cloud account behind your hub.

What components make up my cloud bill¶

There are a few kinds of infrastructure that make up your cloud bill. Here is a short summary:

• Nodes for user sessions: A “node” is kind-of like a virtual machine or a dedicated computer. It is reserved cloud infrastructure that you can use as you wish. Nodes have resources allocated to them (e.g., 100GB of RAM). JupyterHub uses dedicated nodes for user sessions, so more users == more nodes. You generally pay cloud providers by the minute for each node used.

• Storage costs: In order for users to persist their work over time, we must pay for filesystem storage. This is used to store user notebooks and content, data, etc. You generally pay cloud providers by the GB over time.

• Nodes for hub infrastructure: In addition to the cloud nodes for user sessions, there are also nodes to run the JupyterHub and supporting infrastructure to manage user log-ins, do monitoring and reporting of activity, etc.

• Nodes for specialized computing: For hubs that have scalable computing resources like a Dask Gateway, these generally request special nodes on the fly. When a scalable computation is executed, the cloud quickly requests many new nodes to complete the computation, and then removes them when it is done. You pay for the time used for each node during this computation.

There are some other components that go into your cloud bill (e.g., “networking costs”) but these are the major pieces.

You pay for what you request, not for what you use¶

Cloud resources like memory and CPU are reserved the moment a user session starts, whether or not the user fully uses them. A user who picks a large resource profile “just in case” still pays the full cost for that capacity. Encourage your users to start with the smallest resource profile that fits their work, and to scale up only when they actually need to.

User actions that impact cloud costs¶

Cloud costs depend on a few key factors that you and your community has control over. Here we list some major considerations (in decreasing order of importance):

• Base user resources needed: The power and complexity of the user environment is the biggest driver of “base cost per user”. This is largely driven by the amount of memory (RAM) each user needs. See below for a more in-depth explanation.

• Community usage over time: Resources are requested from the cloud “on-demand”, meaning that your cloud costs will scale up and down with number of active users at any given moment.

• User storage over time: User storage is different from on-demand resources, because it’s “always being used” even when you’re not logged-in. We recommend storing large datasets and such in cloud object storage, which is much cheaper.

• Dedicated vs. shared infrastructure: If your community requires their own dedicated cloud infrastructure (for example, a dedicated Kubernetes cluster) then this will boost your cloud costs because you will not be sharing this cost with other communities.

• Cloud optimizations: There are many ways to make cloud infrastructure more efficient and scalable, and the 2i2c engineering team is constantly experimenting with ways to lower costs for communities. For many non-2i2c hubs, inefficiency is a large source of cloud cost, though the 2i2c hubs are already well-optimized.

Estimate my cloud costs¶

The following is a very rough guideline to follow in order to understand and estimate what your cloud costs might be. These are similar whether you’re using 2i2c to manage your hub, or running it yourself.

Generally speaking, the biggest technical driver of cloud costs is user memory (RAM). This is because RAM must be “reserved” on a node, and each node has a finite amount of memory available to it.

Let’s say a user node costs $100.00 an hour, and comes with 100GB total RAM. If each user is guaranteed 1GB of RAM, then the node can theoretically fit 100 users at a time. 100 simultaneous users will cost $100.00 an hour, or roughly $1 / user / hour.

If we double the guaranteed RAM available to users, then the node can now fit 50 users at once (100 GB / 2 GB per user = 50 users total). We now need twice the number of nodes to handle the same number of users. 100 simultaneous users will now cost $200.00 an hour, or roughly $2 / user / hour.

In practice, the cost per node depends heavily on the cloud provider, and is constantly in-flux. To estimate your own cloud costs, follow these steps:

1. Estimate memory available to each user. The amount of RAM needed for each user is often the biggest driver of cloud cost. Decide the “maximum” amount of RAM that a user will generally need, and multiply that by 1.5x.

2. Determine how many average simultaneous users you’d like a hub to support. This isn’t necessarily the total size of your community, but how many people you think will be using the hub at the same time.

3. Look up the monthly price for an n1-highmem-4 node. This is a basic node type that serves most use-cases and can be used as a benchmark for comparison.

   1. Go to the Google Cloud pricing page. This lists prices for many kinds of nodes with Google Cloud Platform.

   2. Go to the N1 high-memory machine types section. This contains prices for all N1 node types with high memory.

   3. Look at the hourly price for n1-highmem-4.

   4. Divide this amount by n_simultaneous_users_per_hour * GB_per_user.

   5. This is your estimated extra cost per hour per user.

4. Estimate storage costs. Estimate your storage costs based on the expected storage each user will take up. 2i2c’s hubs use a standard NFS File Storage for most hubs, which has very fast latency for interactive computing. Here are Google’s file storage prices, for example. You can estimate these costs based on the expected storage used across all of your users.