NIBE Heat Pump Monitoring via NIBE Uplink API

Introduction

I’ve been getting quite a few questions about my use of the NIBE Uplink API for monitoring my NIBE heat pump installation so this Page is a simple HOWTO guide for using a Perl script running on Linux to grab data via the NIBE Uplink API.

Disclaimer

This procedure works for me and I’m publishing it here as a Proof of Concept in case it helps other people trying to accomplish the same objective. It may not work for you. It may not get updated in line with changes to the NIBE Uplink API service. You are responsible for complying with the terms of the NIBE Uplink API Services Agreement and for keeping the application Identifier, Secret and any generated user Tokens confidential.

As with all other material on this Blog this article is published under the terms of the Creative Commons Attribution-ShareAlike 4.0 International License

My Use Case

I have an 8kW NIBE F1145 Ground Source Heat Pump which is configured to send data to the NIBE Uplink service over the Internet. The uploaded data is accessible via the NIBE Uplink website and via the NIBE Uplink smartphone apps, but those apps use the API under the covers and it’s possible to call the API directly from a custom program or script to access all the same data – and then do more interesting things with it.

I have written a script which is called on a timed schedule (every 5 minutes) to download the parameters I’m interested in which I then re-publish using MQTT (my personal preference; in no sense essential) to integrate them with temperature data from other sources (Dallas Semiconductor One-Wire sensors and Oregon Scientific temperature / humidity sensors – see other pages on this Blog for more details of those if you’re interested). Everything then gets loaded into an InfluxDB time-series database from which it can be extracted and graphed with Grafana (again my personal preferences but they work well for me).

An example of the sort of dashboard generated by Grafana is shown below but to be honest a static screenshot doesn’t do it justice since hovering over any of the graphs displays the numeric data values and clicking on one or more of the labels in a legend shows just those lines (and auto-scales to suit). Scrolling left and right across different time periods and zooming in and out is also easy and slick.

NIBE Heat Pump Dashboard using data extracted via the NIBE Uplink API

NIBE Heat Pump Dashboard using data extracted via the NIBE Uplink API

Pre-Requisites

At a bare minimum you will need the following to use this procedure:

  1. A NIBE Uplink account with at least one heat pump registered to it and reporting data which is viewable via the website or one of the smartphone apps.
    • Anyone can use the “Register an account” button on the main NIBE Uplink page to create an account if you don’t have one, but unless you have a heat pump linked to the account the API won’t return any data.
  2. A Linux computer which is able to access the Internet and which will run the Perl script to download data via the API.
    • It doesn’t need to be particularly powerful – a Raspberry Pi or similar single-board-computer would be adequate. It could also be a small Virtual Server (tested successfully with a Microsoft Azure Public Cloud ‘Basic_A1’ size of virtual server).
    • It doesn’t need to run a web browser or have a graphical display attached – remote access via a terminal window using a tool like PuTTY or ‘ssh’ is sufficient.
    • Since this Page is about the NIBE Uplink API and not about installing or configuring Linux or Perl, I’m going to declare that you should use a Linux distribution which has the Perl LWP::Authen::OAuth2 Module available as a standard operating system Package. This is true of Ubuntu Server 16.10, for example (where the package is called ‘liblwp-authen-oauth2-perl’).
    • If you are familiar with Linux and Perl then go ahead and grab the source code for LWP::Authen::OAuth2 and install it on a Linux distribution of your choosing. I have it working fine on Raspbian Jessie Lite on a Raspberry Pi Model B, but I’m not going to document the somewhat lengthy procedure required to achieve that.
  3. An Internet-connected computer with a standard graphical web browser (Firefox, Internet Explorer, Safari etc.) which can be used to complete Steps 1 and 2 in the procedure described below. This can be any platform you like – potentially even a Smartphone – and certainly could be the same Linux computer described at Item 2 above if that meets the requirements, though typically it will be a different machine.
    • If the computer running the browser is not the same as the one running the script, then ensure you can connect to the computer running the script, since you will need to copy-and-paste a long text string between the two, accurately and quickly.
  4. A basic knowledge of the Perl scripting language, editing files and installing software packages on Linux.
    • I used Perl because that’s what I’m most familiar with and it has some add-on modules which help with much of the necessary processing. Alternatives such as PHP, Java or similar can probably be made to work too if that’s what you prefer.

NIBE Uplink API – Documentation and Underpinning Technologies

The official documentation for the NIBE Uplink API is at https://api.nibeuplink.com/docs/v1 (note you need to login to access that page). The detail is described on those pages but in summary:

  • It’s a RESTful API accessed over HTTPS
  • Data is encoded using JSON
  • All requests are authenticated using OAuth2
    • This presents the single biggest challenge because you need to obtain a valid Token and then Refresh that whenever it Expires

Personally I found the biggest barrier to getting the API working was understanding what the OAuth2 Callback URL was all about and what would be an acceptable value to use for that (there are several constraints). Actually it’s not all that complicated and it only needs to be used once, in order to initially get an Access Token. Importantly, it’s possible to run some steps in the procedure on different machines (as long as they’re done in quick succession). Here’s a summary of how it works:

  • When one of your application’s users (most likely yourself in this scenario) enters their NIBE Uplink credentials in a browser window and agrees to grant your application access to their data, the NIBE Uplink website redirects them to the Callback URL, attaching an Authorization Code which is only valid for a limited time and can only be presented once
  • Before the code expires, it’s necessary to present it to the NIBE Uplink Token URL together with other application-specific parameters in order to be allocated an Access Token and an accompanying Refresh Token
  • As long as you can get the Authentication Code presented to the Token URL before it expires, you can do that from anywhere – it doesn’t need to be from the same machine as is hosting the Callback URL though that is typically the scenario described in other OAuth2 documentation

I believe it’s necessary for the Callback URL to use HTTPS and since that can be problematic to configure I’m hosting a simple script on the same server as this Blog which anyone should be able to specify as their Callback URL – it will simply print out the generated Authorization Code (or any error message). That URL is: https://www.marshflattsfarm.org.uk/nibeuplink/oauth2callback/index.php

(I’m going to assume the nice people at NIBE Uplink will permit multiple applications to be registered which have the same Callback URL.)

If you want to host your own installation of the Callback URL the PHP script code I’m using is as follows:

<?
// oauth2callback/index.php

echo "https://www.marshflattsfarm.org.uk/nibeuplink/oauth2callback/index.php ";
echo "OAuth2 Callback Script<br>";

if ( isset( $_GET['code'] ) ) {
    echo "Parameter 'code' found, value is: ";
    $code = $_GET['code'];
    echo "$code<br>";
} else {
    echo "Parameter 'code' not found<br>";
}

if ( isset( $_GET['state'] ) ) {
    echo "Parameter 'state' found, value is: ";
    $state = $_GET['state'];
    echo "$state<br>";
}

if ( isset( $_GET['error'] ) ) {
    $error = $_GET['error'];
    echo $error;
}

?>

The Perl module LWP::Authen::OAuth2 is used to hide a lot of the complexity of OAuth2. There’s a good documentation page for that at http://search.cpan.org/dist/LWP-Authen-OAuth2/lib/LWP/Authen/OAuth2/Overview.pod though in principle if you exactly follow the procedure outlined below you don’t need to understand how it works.

 

Step-By-Step Procedure

Step 0 – Prepare the Linux Computer

The computer which will run the script that calls the NIBE Uplink API needs some preparation to ensure the necessary supporting Perl modules are available. The following assumes it is running Ubuntu Server (or Desktop) 16.10, for the reasons described above.

  1. First ensure the Perl software is installed and check the location of the ‘perl’ executable:
    $ which perl
    /usr/bin/perl

    Amend the scripts below if for some reason this is not as shown.

  2. It’s unlikely the LWP::Authen::OAuth2 Perl Module is already present so proceed to install that, using the following command (which will automatically install all required dependencies or will do nothing if it is already installed):
    $ sudo apt-get install liblwp-authen-oauth2-perl

Step 1 – Register your Application

Start by Registering the new Application you will be creating at the NIBE Uplink website so that you get allocated your various unique ID strings which will be required later.

  1. Connect to https://api.nibeuplink.com/ using your favourite web browser on any machine you like (it does not need to be the machine which will later run the Perl script)
  2. Login with your NIBE Uplink username and password
  3. Click on the MY APPLICATIONS tab
  4. Press the Create application button
  5. Populate the Name, Description and Callback URL fields – choose your own values for Name and Description and either specify your own Callback URL (if you’ve set one up) or use https://www.marshflattsfarm.org.uk/nibeuplink/oauth2callback/index.php
  6. Read the Services Agreement, click to confirm your acceptance of that and then press the Create Application button
  7. Record the allocated Identifier and Secret strings and keep them secret!

Step 2 – Get an Authorization Code

Next, use your Identifier at the NIBE Uplink Authorize Endpoint to get an Authorization Code which relates to your usage (as an end-user) of your application (as a developer).

  1. Compose a URL in your browser’s address field by substituting your own value for [Identifier] (as just allocated) in the following URL template:
    https://api.nibeuplink.com/oauth/authorize?response_type=code&client_id=[Identifier]&scope=READSYSTEM&redirect_uri=https://www.marshflattsfarm.org.uk/nibeuplink/oauth2callback/index.php&state=STATE

    • If you’re using your own Callback URL (redirect_uri) substitute that too
    • The state should really be a random string that only you know and can verify when you see the same string returned, but hard-coding that as the fixed string ‘STATE’ doesn’t hurt for now. The state parameter does appear to be mandatory for NIBE Uplink API calls.
  2. Hit enter to navigate to the edited URL
  3. Assuming you’re already logged in to the NIBE Uplink API website (e.g. having just created your Application) in the same Browser you won’t be prompted for your credentials, otherwise expect to have to enter your Username and Password
  4. Expect to see a page like the screenshot below (you’ll see the Name you specified rather than “Simple API Script”)
    NIBE Uplink API Authorize page
  5. Convince reCAPTCHA that you’re not a robot then press the Accept button
  6. If everything works you’ll see a very simple web page which looks like the following
    Authorization Code
  7. That value for Parameter ‘code’ (pixelized in the screenshot) is your Authorization Code which you’ll need to copy-and-paste at Step 3 – before it expires. Most likely you’ll need to re-run this step to get a new code once you’ve prepared Step 3.

Step 3 – Convert the Authorization Code to an Access Token

Switching now to the Linux computer which will be running your script, you need to create a Perl script which will use the Authorization Code in conjunction with other details (in particular the Identifier and Secret allocated previously) to request an Access Token which can then actually be used when calling the NIBE Uplink API.

  1. Use your favourite text editor to create a file called request_tokens.pl containing the following text:
    #!/usr/bin/perl
    
    use strict;
    use warnings;
    use LWP::Authen::OAuth2;
    
    our $filename = $ENV{ "HOME" }.'/.NIBE_Uplink_API_Tokens.json';
    
    sub save_tokens {
        my $token_string = shift;
        open( my $fh, '>', $filename );
        print $fh $token_string;
        close $fh;
    }
    
    my $oauth2 = LWP::Authen::OAuth2->new(
        client_id => '[Identifier]',
        client_secret => '[Secret]',
        token_endpoint => 'https://api.nibeuplink.com/oauth/token',
        redirect_uri => 'https://www.marshflattsfarm.org.uk/nibeuplink/oauth2callback/index.php',
        request_required_params => [ 'redirect_uri', 'state', 'scope', 'grant_type', 'client_id', 'client_secret', 'code' ],
        scope => 'READSYSTEM',
        save_tokens => \&save_tokens
    );
    
    print "Create a new Authorization Code and enter (copy-and-paste) it here\n";
    my $code = <STDIN>;
    chomp $code;
    
    $oauth2->request_tokens(
        code=> $code,
        state => 'STATE'
    );
    
    exit;
    
  2. Replace the bold strings in square brackets with your own values – retain the single quotes but replace the rest of the text, including the brackets. For example if your Identifier is XYZ, the line should read:
    client_id => 'XYZ',
  3. Change the file permissions so the script is ‘executable’ by the owning user:
    $ chmod u+x request_tokens.pl
  4. Run the script:
    $ ./request_tokens.pl

    and enter a newly-created Authorization Code when prompted (i.e. repeat Step 2 so you have a new Authorization Code to enter)

  5. If everything worked perfectly you will see no output text and you will find you have a new dot-file in your Home Directory called “.NIBE_Uplink_API_Tokens.json” containing the Access Token and an accompanying Refresh Token, serialized using JSON syntax. Note that it typically takes several seconds for the Tokens to be issued – the same delay is evident when using the Smartphone apps.
  6. Protect the contents of the token file with the same care you would a password or SSH Private Key. Make sure it is only readable by the owning user:
    $ chmod 600 ~/.NIBE_Uplink_API_Tokens.json
  7. If instead you see a one-line error message like:
    OAuth2 error: invalid_request at ./request_tokens.pl line 30.

    then the NIBE Uplink system objected to your request for some reason – typically because the Authorization Code had expired or had been presented more than once – so check the input and try again

  8. If you see a lot more error text referring to an Internal Server Error then that’s the NIBE Uplink system responding to an unexpected error – check the Perl script matches the example above and try again

Step 4 – Call the NIBE Uplink API

Now we have an Access Token we can actually call the API, and now things are less time-critical because the Token will be automatically renewed as required.

  1. Use your favourite text editor to create a file called nibe_uplink_api.pl containing the following text:
    #!/usr/bin/perl
    
    use strict;
    use warnings;
    use JSON;
    use LWP::Authen::OAuth2;
    
    # File-scope variables, available to Main Program and Subroutines
    our $filename = $ENV{ "HOME" }.'/.NIBE_Uplink_API_Tokens.json';
    
    # Subroutine to save Access and Refresh tokens when those change
    sub save_tokens {
        my $token_string = shift;
        open( my $fh, '>', $filename );
        print $fh $token_string;
        close $fh;
    }
    # Main Program
    
    # Local variables
    my $token_string;
    my $url;
    my $response;
    my $decoded;
    
    # Read saved token_string from file
    open( my $fh, '<', $filename )
        or die "Could not open file $filename: $!";
    $token_string = <$fh>;
    chomp $token_string;
    close( $fh );
    
    # Construct the OAuth2 object
    my $oauth2 = LWP::Authen::OAuth2->new(
        client_id => '[Identifier]',
        client_secret => '[Secret]',
        token_endpoint => 'https://api.nibeuplink.com/oauth/token',
        redirect_uri => 'https://www.marshflattsfarm.org.uk/nibeuplink/oauth2callback/index.php',
        request_required_params => [ 'redirect_uri', 'state', 'scope', 'grant_type', 'client_id', 'client_secret', 'code' ],
        scope => 'READSYSTEM',
        token_string => $token_string,
        save_tokens => \&save_tokens
    );
    
    # Make a simple NIBE Uplink API call - Get the Systems for this Account
    $url = "https://api.nibeuplink.com/api/v1/systems";
    $response = $oauth2->get( $url );
    if ( $response->is_error ) { print $response->error_as_HTML }
    if ( $response->is_success ) {
        $decoded = decode_json( $response->content );
        # Expect $decoded to be a Reference to a Hash
        my $objects = $decoded->{ 'objects' };
        # Expect $objects to be a Reference to an Array of Hashes
        for my $hashref ( @ { $objects } ) {
            my $system = $hashref->{ 'systemId' };
            print "systemId = ".$system."\n";
        }
    }
    
    exit;

    That simple line $response = $oauth2->get( $url ); is doing most of the work here. Under the covers it’s taking care of presenting the Access Token when making the API call, checking whether the Token has expired and automatically renewing it if necessary. Whenever it gets renewed the new Token gets written back to the .json file so it is available for next time.

  2. Change the file permissions so the script is ‘executable’ by the owning user:
    $ chmod u+x nibe_uplink_api.pl
  3. Run the script:
    $ ./nibe_uplink_api.pl

    If it works as expected it should list out the systemId number(s) of the heat pump(s) assigned to your NIBE Uplink account

 

JSON Response Formats

Some people have asked for examples of the JSON strings that are returned in response to the NIBE Uplink API calls. I’ve replaced strings like the serial number of my heat pump and my postal address details with XXXXXX’s but otherwise the responses are exactly as returned.

{"page":1,"itemsPerPage":30,"numItems":1,"objects":[{"systemId":XXXXX,"name":"F1145-8, 1x230","productName":"NIBE F1145","productImage":{"name":"NIBE_F1145","sizes":[{"width":75,"height":100,"url":"/Content/Products/F1145_75x100.png"}]},"securityLevel":"ADMIN","serialNumber":"XXXXXXXXXXXXXX","lastActivityDate":"2017-04-17T14:12:41Z","connectionStatus":"ONLINE","address":{"addressLine1":"XXXXXXXXXXXXXXXXX","addressLine2":"XXXXXXXXXX","postalCode":"XXXXXXXX","city":"XXXXX","region":"XXXXXXXXXX","country":"UNITED_KINGDOM"},"hasAlarmed":false}]}
[{"categoryId":"STATUS","name":"status","parameters":null},{"categoryId":"CPR_INFO_EP15","name":"compressor module ","parameters":null},{"categoryId":"SYSTEM_1","name":"climate system 1","parameters":null},{"categoryId":"ADDITION","name":"addition","parameters":null},{"categoryId":"AUX_IN_OUT","name":"soft in/outputs","parameters":null},{"categoryId":"SYSTEM_INFO","name":"info","parameters":null}]
[{"parameterId":40004,"name":"40004","title":"outdoor temp.","designation":"BT1","unit":"°C","displayValue":"9.8°C","rawValue":98},{"parameterId":40067,"name":"40067","title":"avg. outdoor temp","designation":"BT1","unit":"°C","displayValue":"8.3°C","rawValue":83},{"parameterId":40013,"name":"40013","title":"hot water top","designation":"BT7","unit":"°C","displayValue":"41.4°C","rawValue":414},{"parameterId":40014,"name":"40014","title":"hot water charging","designation":"BT6","unit":"°C","displayValue":"37.9°C","rawValue":379},{"parameterId":43005,"name":"43005","title":"degree minutes","designation":"","unit":"DM","displayValue":"-57DM","rawValue":-578}]

 

Next Steps

If this procedure works for you, great! Please leave a Comment to let me know. If it doesn’t work or doesn’t makes sense please also leave a Comment and I’ll try to help.

Clearly just printing out the systemIds isn’t very interesting but the NIBE Uplink API Documentation describes the other API calls in some detail.

A slightly sanitized version of the script I actually use can be downloaded from here. Note it has dependencies on further Perl modules and won’t work without other pre-requisites being satisfied. However, it does provide examples of calling some of the other NIBE Uplink APIs and how to extract the details from the slightly convoluted nested data structures.

 

Further Reading

I found the following resources useful for understanding how to get OAuth2 working:

CC BY-SA 4.0 NIBE Heat Pump Monitoring via NIBE Uplink API by Marsh Flatts Farm Self Build Diary is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

6 thoughts on “NIBE Heat Pump Monitoring via NIBE Uplink API

    • Thanks for the feedback; I *knew* there was something else I had meant to add to the article but I’d forgotten it was to do exactly that.

      PHP script source now added to the body of the article.

      David

  1. Hi!
    First of all, nice work, very interesting and I have learned a lot from it.
    Unfortunately I have not yet any Nibe geothermal heat pump, but plan to buy one.
    I’m trying to do the very same as you but in my case I will use Java.
    So my question is, if it will be possible for you to just send me some info. about the actual JSON respons that you use in your script. If I do have the JSON syntax will it be possible for me to do some coding before I have my very own pump and regarding to that also access to my own account. I see that you have an F1145, I’m quit not sure yet which model that will be the best for me, but I assume that they have similar syntax regarding JSON-respons.
    Thank’s in advance!

    Regards Örjan

  2. For the record, I’ve exchanged a few emails with Örjan and discovered they’re planning to develop a NIBE heat pump binding for the OpenHAB home automation framework, which I also use.
    There’s a discussion thread on the OpenHAB forum: https://community.openhab.org/t/nibe-uplink-binding/25756
    I’ve also added some JSON Response Formats examples within the body of this Page.

  3. Just to fill in with another work going on if more people find think link when searching for NIBE API, which I did =).
    There is now a working custom component to http://www.homeassistant.io that was done by elupus (https://github.com/elupus) that works great! Everything you get from NIBEs App you can now see in your Home Assistant setup.

    You can get the info here:
    https://github.com/elupus/hass_nibe
    https://community.home-assistant.io/t/nibe-uplink-api-component/18173

Leave a Reply

Your email address will not be published. Required fields are marked *