RosettaCloud™ Big Data
Spark and Hadoop on demand
Pay your cloud provider.
Once.
Analytics platforms tend to run a meter of their own alongside the one your cloud provider is already running, and to ask you onto a runtime you cannot take with you. Neither is necessary to get a Spark cluster when you need one.
What it changes
One meter, not two
Compute is billed by your cloud provider at your rates. Nothing is metered on top of infrastructure you are already paying for.
Nothing to port onto
Standard distributions and the applications you pick. Jobs written against open Spark and Hadoop run unchanged.
The cluster ends with the job
Described once and launched on demand, so there is no reason for one to become permanent.
Your data never moves
The cluster is built next to the data, in your own account. Nothing is copied out to be processed somewhere else.
The cheapest platform to leave
is the one you never converted to
A managed analytics platform usually asks you to adopt its notebooks, its runtime and its way of describing a job. That work is invisible until the day you want to move, and then it is most of the cost of moving.
The release you choose
Pin the distribution version and pick the applications your jobs expect, rather than accepting whatever a vendor runtime bundles this quarter.
The environment you built
Your bootstrap script and your own image, so the libraries and configuration your team depends on are there before the first job runs.
The code you already have
Open Spark and Hadoop, so jobs run as written. Nothing is rewritten to arrive, and nothing has to be rewritten to leave.
Pick the tools.
Not the setup.
A cluster reachable only over a terminal is a cluster only some of your people can use. Choose the applications you want when you describe the cluster, and they are running on it the moment it exists.
Notebooks and a desktop
Jupyter, RStudio, a terminal or a full graphical desktop, opened in a browser against the cluster rather than over a connection somebody has to set up first.
Several of them at once
You are not picking one. Choose the set the work needs and move between them, all against the same cluster.
Two people in one session
A collaborative RStudio session takes more than one person, so working through a failing job becomes a conversation rather than an exchange of screenshots.
Your own tools alongside them
Your container images, startup scripts and environment, so an in-house tool arrives with the standard ones instead of being installed afterwards.
Nothing is installed by hand, and nothing has to be installed again when the cluster is replaced next month.
It comes up connected
Most of the time between having a cluster and using one goes on the work between the pieces. That part is done for you, identically on every cloud.
Storage is already mounted
Attach the storage a cluster needs and it is there at its path when the cluster comes up. Nobody edits a mount table, copies data in, or explains to the next person where the files went.
There are no keys to hand out
Cloud keys are held by the platform, not by the people using it. Nobody is emailed a private key, nobody stores one on a laptop, and there is nothing to rotate or to leak.
Certificates are issued and attached
A service that should be reachable over HTTPS comes up that way, without a certificate request, a renewal reminder, or an expiry nobody noticed until it broke.
It has a name, not an address
DNS records are created on the domains you have mapped, so people reach the cluster by a name that means something and keeps working when the machine behind it changes.
And you can share it while it is running
A live cluster is shared with a colleague, a group or a whole organisation the way a document is. They get access to the thing itself, not a copy of it and not a rebuild of it, and that access ends when the share does.
See how sharing works →Long-lived clusters are
where the money goes
Clusters become permanent because standing one up is hard, and then they are paid for whether or not anything is running on them. Take away the difficulty and the permanence goes with it.
Spare capacity where it counts
The head node and the worker nodes are priced separately, so the head node stays on demand while the workers, which are most of the bill, run on spare capacity.
Forgotten clusters get found
An idle cluster is detected alongside idle machines, databases and warehouses, and reported before anything is switched off.
The budget applies at launch
A cluster the account cannot afford is refused at the cloud’s own API, and what does run is attributed to the person and project that started it.
Your accounts, your rates,
your commitments
Clusters are built in your own cloud accounts and billed to you by the provider. Discounts you have negotiated still apply, commitments still draw down, and the data stays where it already lives.
Common questions
Do we pay anything on top of our cloud bill for the compute?
No margin is added to infrastructure. Clusters run in your own cloud accounts and your provider bills you directly at the rates you have already negotiated, including any commitments or credits you hold. There is no second per-unit meter running alongside the cloud one.
Will our existing Spark jobs run without changes?
Yes. These are standard distributions with the applications you choose, so jobs written against open Spark and Hadoop run as they are. There is no proprietary runtime to port onto, which also means there is nothing to port off later.
How do we stop paying for a cluster between jobs?
Launch it for the work and let it go when the work is done. Clusters are described once and started on demand, so the usual reason they become permanent, that standing one up is painful, no longer applies. An idle cluster left running is detected alongside other waste.
Can we run on spare capacity without risking the run?
Yes, and selectively. The head node and the worker nodes take separate bid prices, so the head node can stay on demand for stability while the workers, which are most of the cost, run on spare capacity.