Webinar: Using Big Data To Enable Smarter Energy

Data Centre Image ISO9001

February 24th, 2016 at 10am GMT/ 11am CET

Source: Hortonworks

Open Energi are presenting a live webinar on the 24th February 10am GMT, it’ll also be available and on-demand immediately below.

Energy wasted through excess capacity is one of the world’s most challenging problems especially in developed countries where energy doesn’t come cheap. Meet Open Energi, an organization working with businesses in the UK to harness excess capacity from their equipment and aggregating it to create a virtual power station to better meet demand. Join this webinar to learn how Open Energi uses data to fuel its virtual power station enabling them to provide cheaper and efficient uses of energy across the UK powered by the Hortonworks Data Platform.

Topics covered:

Exploiting Excess Energy Capacity
Building a Virtual Power Station
Details of the Big Data platform driving the future of energy consumption

Mapping Britain’s Heat Storage Potential

Bitumen tanks

Chris Kimmett, Commercial Manager, Open Energi

The energy system is undergoing a huge transformation away from centralised generation to small-scale, distributed power. National Grid’s Future Energy Scenarios (FES) models indicate that by 2020, small-scale, distributed generation will represent a third of total capacity in the UK and, as a result, speed of response to changes in energy supply and demand will be more important than ever.

And it is not only the increase in distributed generation that will prove challenging for the UK grid. The coal-fired Ferrybridge, Longannet, Fiddler’s Ferry and Rugeley are all expected to come offline this year, and with gas power stations procured under the Capacity Market now in doubt, the cushion between supply and demand is smaller than ever.

A new source of flexibility is urgently required, and storage to provide this flexibility will be an increasingly essential part of a responsive, secure and sustainable energy future for the UK.

Energy storage is commonly understood to mean batteries and pumped hydro systems. While both are valuable, current costs, installation times, and issues around recycling and decommissioning are all prohibitive to wider deployment. But storage exists in a number of forms, including through demand side response (DSR), which takes advantage of latent heat in energy-intensive equipment and devices to create new flexibility for the grid.
If too much energy is supplied at any given time, it doesn’t have to be stored in a battery: instead, Internet of Things (IOT) based forms of demand response can adjust the consumption of energy-intensive devices to make use of power when it is available. In instances when there is not enough power, demand can be deferred rather than drawing from a battery to supplement supply.

This smart DSR approach is ideally suited to heating and cooling assets that have the characteristics of stored energy devices.  By harnessing existing everyday equipment, from fridges to furnaces, and invisibly switching them on or off for a few minutes at a time, energy demand can be adjusted to meet available supply in real-time, creating a distributed storage technology.

Take the asphalt plants which manage the complete asphalt production process for road construction as an example. Liquid bitumen for road surfacing is stored in large, well-insulated tanks, and a heater maintains the temperature of the bitumen between a low set point (typically 150 degrees C) and a high set point (typically 180 degrees C).

These tanks have “thermal inertia”, meaning the amount of energy they use can be adjusted and the temperature of the bitumen won’t be immediately affected: Bitumen tanks can be switched off for an hour and the temperature may only fall by between 0.5-15 degrees C.

Using demand response technology, bitumen tanks can deliver a full response to National Grid within two seconds (quicker than traditional thermal generation) and for up to 30 minutes, provided they are within their set-points. The average duration of Open Energi’s switch requests to bitumen tanks is just 3.3 minutes.

Cooling systems such as supermarket refrigeration also provide a distributed storage network that can help to balance UK-wide electricity supply and demand in real-time.

Open Energi estimates that if Dynamic Demand was deployed in the commercial refrigeration assets of the five largest retailers in the UK, it could meet approximately 6% of the UK’s total 1.8GW requirement for Frequency Response, roughly equivalent to 100 MW. This would generate revenues of up to £10 million a year for the asset owners and reduce UK CO2 emissions by around 227,600 tonnes a year.

Other latent heat storage assets include: heating, ventilation, air conditioning, and hot water boilers in commercial property; electric induction furnaces, ovens and melting pots in foundries and metal processing sites; and heaters and aerators at water processing sites.

Because these devices have already been built, it is possible to aggregate the stored thermal energy they contain and build a virtual power station at a fraction of the cost of building a grid scale battery or new generation capacity. The capital cost of building a new peaking power station can be up to £5 million per MW and battery systems in the region of £0.5 million-£1.8 million per MW. A MW of demand response, on the other hand, costs around £200,000 to aggregate.

DSR, coupled with on-site generation and energy storage technologies means that the energy market is no longer a linear value chain driven by fossil fuel production but is becoming decentralised and bi-directional; creating a new energy economy where energy consumers can both take and provide service back to the grid and generate revenue.

To realise the full potential of DSR technology we now need to further understand where the potential for flexibility, including latent heat, lies across the UK’s entire electricity network: assessing both regions and sectors.

In the same way that traditional energy commodities like oil, gas and coal are mapped by geologists to identify resource rich areas, a flexibility mapping process will enable demand response aggregators to identify the DSR ‘hot spots’. This in turn will give business, industry and policy makers the confidence to invest in DSR technology ahead of building additional spinning reserve, and the certainty they need to plan for a future where flexible Demand Response plays an integral role in delivering a secure and resilient energy system.

By using land data from regional authorities, for instance the GLA for London, the industry can develop a better understanding of where the flexibility potential lies, whether that be in heavy industry, commercial buildings or residential areas.

Open Energi is working to map flexible demand in the UK from the bottom up, asset by asset, sector by sector, to model the capacity in the market and demonstrate how much generation can be displaced.

Increasing flexibility on the grid has historically meant building more generation, but latent heat in energy intensive equipment presents a hugely valuable opportunity. And through mapping, this opportunity can be realised at scale for the UK.

Powering a Virtual Power Station with Big Data

Tech image

Michael Bironneau, Data Scientist, Open Energi

At Open Energi, we think of our service as an automated, virtual power station. Whenever the electric grid experiences sudden, unforeseen surges in supply or demand, assets under the control of our Dynamic Demand algorithm automatically pick up the slack – just like a power station would but cheaper and cleaner.

In order to prove that we’ve delivered this service and keep it running at optimum, we need to analyse large amounts of data relatively quickly. We’re also making our service smarter so that more assets will be able to participate in Dynamic Demand than before. This is where Big Data and Hortonworks Data Platform come in.

Big Data is a phrase that has been floating around companies like Google for the last two decades. It has never really had a precise definition, but when used colloquially it usually means that someone somewhere is running out of space for your data and/or computing power to analyse it. Sometimes it also means that your data is such a mess that it will take a sentient super-robot to make sense of it (thankfully, that’s not our case).

In the context of Dynamic Demand, data is our greatest asset: it tells us when we’ll be able to turn a certain asset on or off without disrupting its primary function, which could be critical to an industrial process. It also allows us to prove to National Grid that an asset we claimed participated in Dynamic Demand actually modified its power consumption to help balance the grid. We want to do more with our data: understand our portfolio of assets better, reduce integration difficulties with assets and accept new assets into our portfolio that don’t meet certain technical requirements. This means running more analyses on live streams of data and integrating many additional datasets together – for us, these problems are Big Data because we can’t cost-effectively tackle them at scale with our current systems.

Hortonworks Data Platform (HDP) is a collection of open source software built on and around Apache Hadoop designed to deal with Big Data, developed partly by large companies like Yahoo and eBay, and partly by the community. Hortonworks’ added value is in the way the tools are seamlessly configured to work as one and the support they provide. They staff core contributors of various projects and technical experts, so help is rarely more than a couple of emails away.

One of the most widely known tools in the HDP toolbox is called Apache Hive. This tool excels at integrating different types of data and allowing them to be queried as one, spreading out the computational cost of the query on as many machines as we can get our hands on. We’re planning on using this for most of our ad-hoc analysis and some batch jobs. Because it is easy to extend with custom logic, we can program Open Energi analytics straight into it. For example, we can call into our Python code which contains functions we may want to evaluate over various pieces of data, without worrying about how that data is stored or whether it is even coming from a single source. Because we deal with a lot of timeseries data we also need to perform resampling operations on a regular basis – these can be painful in regular query languages such as SQL but Hive’s extensibility makes it a breeze.

For our low-latency applications, Hortonworks package a piece of software called Apache Storm. This software is designed to run an entire graph of computations on a live stream of data, adding reliability in case parts of the graph fail. For example, when a device sends us a power reading, we can correlate it with Dynamic Demand state, train machine learning models and update a cache powering a live dashboard – all without leaving Storm.

Hortonworks take security very seriously and so do we. HDP comes with tools such as Apache Knox and Apache Ranger that deal with questions of who should have access to which piece of data. Even though enterprise-grade security is a relatively new concept in the world of Big Data, HDP has fully caught up and the core systems now support transparently encrypting data in movement and at rest, with central management in Ranger allowing us to effortlessly define security policies that comply with our business requirements and the high standards that are expected of us.

For future use, we’re excited about Apache Kylin, a brand new piece of software originally created by eBay. While not yet part of HDP, it is built on the same software ecosystem and can easily be integrated. Kylin allows for a different type of data modelling using metrics (or KPIs) and dimensions (eg. “client”, “date” or “type of load”). Roughly speaking, the engine stores pre-computed aggregates of the metrics over the space spanned by the various dimensions. For example, suppose a metric is “power consumption” while the dimensions of interest are “time” and “asset type”. Kylin could return answers almost immediately to questions such as “what was the mean power consumption last month for all assets of type ‘bitumen tank’?” At Open Energi we have many KPIs we want to keep track of and drill into – this is something Kylin should be excellent at managing for us.

Finally, we’re looking into Hortonworks’ new product called Hortonworks DataFlow (HDF). Originally created by the NSA, then integrated into the Apache Foundation’s portfolio under the name Apache Nifi, this project does what it says on the tin: it helps create and manage flows of data. It solves many technical problems, such as what to do when one component in the dataflow can’t keep up with the volumes of data it’s receiving, or how to prioritise which data to send at a given time. While our bespoke systems already solve many of these issues, HDF can do more, such as querying data that lives on individual devices without them ever having to send the raw data back. We’re always looking for ways to get more out of ephemeral data that lives on assets in the field but never gets sent back to our database, so we’re looking forward to trying this out.


Tarmac Harper Lane image

Cras tristique turpis justo, eu consequat sem adipiscing ut. Donec posuere bibendum metus.

Continue reading

Ever ready: will batteries power up in 2016?

Open Energi Banner ADE

David Hill, Business Development Director, Open Energi

Open Energi tends to extol the virtues of Demand Side Response as a solution to the energy storage challenge.  It provides a no-build, sharing economy approach which is cheap, sustainable, scalable and secure.

By harnessing flexible demand and tapping into the thermal inertia of bitumen tanks or the pumped energy stored in a reservoir for example, we have created a distributed storage network able to provide flexible capacity to the grid in real-time without any impact on our customers.

But flexibility comes in many forms, and as the cost of energy storage systems tumble, it looks like 2016 might be the year when commercial batteries become a viable part of the UK’s electricity infrastructure, with recent analysis suggesting they could deliver 1.6GW of capacity by 2020, up from just 24MW today.

The price of energy storage systems is expected to fall sharply over the next three decades, with Bloomberg New Energy Finance predicting the average cost of residential energy storage systems will fall from $1,600 per KWh in 2015 to below $1,000 per KWh in 2020, and $260 per KWh in 2040.

As costs have fallen we have seen increasing interest from industrial and commercial customers keen to explore the benefits of installing batteries on-site and looking at systems capable of meeting 50%-100% of their peak demand – depending on their connection agreement (although it is worth noting an export licence is not a prerequisite).

In addition to providing security in the event of power outages, battery systems can help companies to reduce their demand during peak price periods, enabling them to seamlessly slash the astronomical costs – and forecasting difficulties – associated with Triads, and minimise their DUoS Red Band charges.

When they aren’t supporting peak price avoidance – which may be only 10% of the time – batteries can help to balance the grid – earning revenue for participating in National Grid’s frequency response markets. For example, discharging power to the system if the frequency drops below 50 Hertz and charging when the frequency rises above 50 Hertz.

National Grid’s new Enhanced Frequency Response market has been developed with battery systems in mind – requiring full response within 1 second – but isn’t expected to be up and running for a year or more.

In the meantime battery systems can generate significant revenues today via National Grid’s Dynamic Firm Frequency Response market, tendering alongside loads from companies like Sainsbury’s, United Utilities and Aggregate Industries, to help balance the grid, 24/7, 365 days a year.  And in the longer term the opportunity exists for companies to trade their batteries’ capacity in wholesale electricity markets.

With these saving and revenue opportunities in mind, we’re now at a point where battery systems can be installed behind-the-meter and deliver a ROI within 3-5 years for industrial and commercial sites. The ROI will be subject to certain factors, such as geographic location, connection size and of course the cost of the battery system itself, but these figures would have been unthinkable only a few years ago.

There are important technical factors to consider, including both the battery sizing in terms of its kW power rating and kWhr energy storage capacity, and also the underlying battery chemistry.  By taking into account the physical location of the battery along with models of different markets that it will operate in, it is possible to narrow down to the most appropriate technical parameters.  Another consideration is the gradual effect of wear and tear on the battery with continuous usage.  By analysing these effects it is possible to reduce some of the uncertainty around battery lifecycles (likely to be in the region of 10 years) and get better predictions of the likely revenue in each year of operation.

But whilst a payback of 5 years seems reasonable from an energy infrastructure perspective (where 15-20 years is more typical) for most companies used to a ROI within 2-3 years on energy projects it is not easy financing battery systems.

Some larger, capital rich companies may have the appetite and money to finance these projects themselves, but the majority of the companies we are talking to are keen to take these assets off balance sheet and finance installations via banks and other investors under third party ownership.

In these circumstances, managing the performance of battery systems – so that they meet their warranty and their lifecycle is maximised – whilst optimising their potential as a flexible resource able to cut energy costs, earn revenue and deliver a vital uninterruptible power supply  during outages will be key to their commercial success and scale of deployment.

The Challenges of Device Management in Energy

The Challenges of Device Management in Energy

1248 CEO Pilgrim Beart chats with David Hill, Business Development Director at Open Energi.

P: “David, tell us a bit about what you’re trying to achieve at Open Energi”

D: Open Energi is deploying ‘Internet of Things’ software within electricity consuming assets and paving the way for a new energy system; a system that is cleaner, cheaper, more efficient and more secure than the system we have today.

Our software is able to measure and monitor machine and appliance behaviour in real-time and subtly adjust electricity consumption in response to signals from the market, preventing fossil fuelled power stations from coming online and maximising our use of renewable energy, without any impact on consumer living standards or business productivity.

Every appliance or machine that we connect to using our Dynamic Demand software is another step towards removing power stations from the electricity system all together – and the money that would have been paid to those power stations is instead paid to British businesses.

P: “That sounds like a classic IoT application to me – making better use of a resource, with benefit to all the parties concerned – and helping address climate change too. What stage have you’ve reached in this ambitious goal?”

D: To date we have signed over 25 customers and deployed our software within over 1230 individual assets across 275 sites. Appliances range from Heating, Ventilation and Refrigeration Appliances in Sainsbury’s, to Water Pumps and Waste Management Equipment in United Utilities. The total power of all the devices that Open Energi interacts with is about 45MW, of which we currently bid about 15MW – around 30% – into energy markets operated by National Grid.

30% represents what is typically flexible at any given moment in time. The remaining power is being used for its primary purpose, e.g. heating supermarkets or pumping water. It is only by being able to identify flexibility on a second by second basis that we are able to provide this service.

P:”And can you give us some idea of the sort of challenges that you’ve encountered as you’ve grown in scale?”

One of the key challenges we face is being able to interface with different industrial and commercial processes – ranging from water pumps to bitumen storage tanks to refrigeration systems.  These systems operate in very different ways and generate different kinds of data, so a key challenge for us is being able to collect and view this data in a coherent way, and then understand how all these different processes are consuming energy. 

Overcoming this problem is an essential part of how we can provide a consistent and reliable service to National Grid from many thousands of individual assets.  Today many aspects of our interface with each of these processes inevitably become bespoke, which then leads to challenges in how we maintain and operate the software on all of our devices, particularly as we scale up.

P: “Your growing need for device management rings very true for us – it’s a common story as a connected service starts to achieve scale. We encountered something similar at AlertMe as we scaled from 1,000 to 10,000 devices, which is why we decided to focus on that challenge at 1248. Can you put some more meat on the bones for us?”

A key area for us is being able to simplify how we connect and exchange information with so many different control systems and processes.  Standards could play a key part here – both in terms of how we connect to other devices and what information we exchange.  This would ease the process of connecting and maintaining devices as we scale up and allow us to focus on getting the maximum value out of the energy flexibility that we’re tapping into.

For example, in the future we believe that all appliances with the ability to provide flexibility to the electricity market will come with those principles built in off-the-shelf, so that an air conditioning controller designed to maintain room temperature is every bit as used to responding to electricity market signals as it is to temperature, likewise with water pumps and bitumen tanks.

Device management is very important when trying to operate service based on very large numbers of connected devices; we can only understand and quantify energy flexibility if we have an accurate understanding of which devices are online, and whether they are functioning correctly. Moving from the world we are in now, to the future I just outlined, will need collaboration with companies like 1248.

P: “David, thanks so much for sharing this with us.”

This blog post was originally featured on www.1248.io

The alternative to a power station has already arrived

Green Fields with sunrise - green energy
Transforming the UK’s energy future

Demand side response and energy storage technologies can unlock power in our energy system.

COP21 is focused on how to generate more and cleaner energy to power a low carbon world. But what if we also set about dramatically changing how we use the energy already available to us? Our creaking systems of power generation have in-built flexibility that needs unlocking so we don’t have to always turn to the option of building another expensive power plant.

Looking back to the time of Thomas Edison, we have made only incremental improvements in actually using the infrastructure we use to generate energy. With only a 6 per cent increase in load factor in 130 years, focusing on building more power stations alone is not a smart solution.

Instead, Open Energi is rolling out invisible and automated technology that allows large industrial consumers to align their energy use with available supply. If just 5 per cent of peak demand is met with flexible power, the response would be equivalent to the generation of a new nuclear power station.

With traditional power infrastructure, it has been especially difficult to store electricity. Today however, batteries are fast becoming a megatrend- allowing consumers to store low cost and renewable energy and deliver it back at times of peak demand. People don’t look out their window to see if the sun is shining or the wind is blowing before putting their kettle on, which means we need a new way of making electricity flexible. Arenko is building a portfolio of energy storage assets which puts the power back into the consumer’s hands whilst providing reinforcement to the electricity grid and reducing the need for significant and costly grid upgrades.

With the exciting growth of new technologies for managing demand it is important not to forget that people are at the heart of the energy system. Energy tech start-up Open Utility is trialling its first service, called Piclo, with Good Energy. The 100 per cent renewable energy marketplace connects wind, solar and hydro generators with businesses that want to cut their carbon emissions by buying clean energy. We’re moving towards democratised and decentralised energy systems, and we need platforms that let people take part. We’re moving away from large power stations, to people generating energy in their back yards, and selling to their neighbour, local school, hospital or workplace. International targets and pledges for cutting carbon emissions can seem quite abstract and unconnected to the everyday person. The advent of an online marketplace puts people back into the equation, giving them the choice to buy local green energy – and play their own part in combating climate change.

With the energy system already transitioning away from traditional models dominated by power stations, Open Energi, Arenko and Open Utility are pioneering a consumer-led, decentralised approach- providing flexible capacity which is exponentially cheaper than the alternatives.

By Open Energi’s David Hill, Arenko’s Rupert Newland and Open Utility’s James Johnston

Demand Response: how much is your flexible demand worth?

David Hill Headshot

Our energy system is changing and this is creating new opportunities for businesses to turn their energy use into an asset. By harnessing flexibility  in your electricity consumption, demand response can provide a new source of income and help build a cleaner, more secure and affordable energy system. Open Energi’s Business Development Director, David Hill explores how energy and technology are converging and putting you in charge of transforming our energy future.

The IOT technology meeting the UK’s grid balancing needs faster than a power station

Tech image

Chris Kimmett, Commercial Manager, Open Energi

It’s a well reported fact that electricity margins are tighter than they have been for a number of years and, as we move towards winter, talk will increasingly turn to the need to balance energy supply and demand in order to mitigate the risk of power black outs in the UK.

Almost all of the UK’s grid balancing has traditionally been done by coal and gas. But the EU’s Large Combustion Plant Directive has limited running hours at a number of plants and in the past twelve months both Longannet and Eggborough power stations, which currently provide around 5% of the UK’s capacity, have announced they will be closing their doors in 2016.

Add to this the solar and wind explosion – by the end of 2015 experts predict the UK will have 10GW of solar power, a benchmark most thought wouldn’t be reached until 2020, and by 2020 National Grid’s Future Energy Scenarios indicate that small-scale, distributed generation will represent a third of total capacity in the UK. This considered, we see that tomorrow’s electricity landscape will look very different to that of today.

The transformation of the energy system away from centralised generation to small-scale, distributed power means that speed of response to changes in energy supply and demand will be more important than ever.

Indeed, while most people are focusing on the tight capacity margin between supply and demand, the real blackout threat could come from generators being unable to respond within the required window to balance instantaneous shifts on the grid.

For more than 12 months, energy data analysts at EnAppSys have been monitoring grid frequency and analysing large deviations which, if not managed, can lead to instability. EnAppSys’ director Paul Verrill says that while we need to ensure the system has sufficient supply to meet demand, the real risk of blackouts could come from this second issue that often falls under the radar: a lack of capacity able to deliver additional power within the required timeframe.

Grid agility and flexibility will be essential as we move away from models of centrally dispatched generation, and National Grid, via its Power Responsive campaign, has already asserted that demand side response (DSR) will play an increasingly vital role in building a resilient, sustainable and affordable electricity system for the future.

This is especially pertinent given the results of new research by Open Energi, National Grid and Cardiff University, which suggests that smart demand side response (DSR) technology can meet the UK’s crucial grid balancing requirements faster than a conventional power station.

The latest research paper, which forms part of the ongoing collaborative research programme between Open Energi, National Grid and Cardiff University, titled Power System Frequency Response from the Control of Bitumen Tanks, looks at the feasibility of DSR to provide a significant share of frequency balancing services.

To test the scale of the opportunity for industrial heating loads to provide frequency response to the power system, bitumen tanks (which contain the glue that binds our roads together) equipped with Dynamic Demand technology were tested in combination with National Grid’s model of the GB transmission system.

Dynamic Demand deployed at scale is able to contribute to the grid frequency control in a manner similar to, and, crucially, faster than that provided by traditional peaking power generation – not to mention more cleanly and cheaply.

Field tests showed that full response could be provided in less than two seconds, as compared to 5 – 10 seconds for a thermal generator. Large scale deployment of Dynamic Demand will reduce the reliance on frequency-sensitive generators and ensure that the grid stays balanced in a cost-effective, sustainable and secure manner.

The research simulations help to shape National Grid’s understanding of DSR as a replacement for frequency-sensitive generation and will be used when they are planning their requirements for grid network operation in the future – with huge impacts on the future of our energy mix.

When launching Power Responsive, National Grid CEO Steve Holliday said: “The move to a low carbon economy coupled with rapid advances in technology and innovation are transforming our electricity supply. But supply is only half the story. The challenge now is to exploit new opportunities to radically evolve our energy system by changing the way we use electricity.”

And this is why the research findings are so significant.

With more renewables and decreased thermal generation, ‘inertia’ on the Grid will decrease, making frequency more unstable. To counteract this effect we need faster response, so by rolling out Dynamic Demand today we are future proofing the Grid.

With their new Power Responsive campaign, National Grid has recognised the need for a new source of flexibility and have stated they are committed to scaling the smart DSR industry.

Demand side response is intelligent energy usage. By knowing when to increase, decrease or shift their electricity consumption, businesses and consumers will save on total energy costs and can reduce their carbon footprints. It is the smart way to create new and efficient patterns of demand.