Azure IoT Suite

Azure Cosmos DB – TTL (Time to Live) – Reference Usecase

October 9, 2018 .NET, .NET Core, .NET Framework, Analytics, Architecture, Azure, Azure, Azure Cosmos DB, Azure Functions, Azure IoT Suite, Cloud Computing, Cold Path Analytics, CosmosDB, Emerging Technologies, Hot Path Analytics, Intelligent Cloud, Intelligent Edge, IoT Edge, IoT Hub, Microsoft, Realtime Analytics, Visual Studio 2017, VisualStudio, VS2017, Windows No comments

TTL capability within Azure Cosmos DB is a live saver, as it would take necessary steps to purge redudent data based on the configurations you may. 

Let us think in terms of an Industrial IoT scenario, devices can produce vast amounts of telemetry information, logs and user session information that is only useful until we operate on them and take action on them, to be specific up to finate period of time. Once that data becomes surplus, we need an application logic that purges these old records.

With the “Time to Live” or TTL, Microsoft Cosmos DB provides an ability to have your documents automatically purged from database storage after a certian period if time(which you configured)

  • This TTL by default can be set on a document collection level and later can be overridden on a per document basis.
  • Once the TTL is set, Cosmos DB service will automatically remove the documents when its lifetime is over.
  • Inorder to track TTL, Cosmos DB uses an offset field to check when it was last modified.  This field is identifiable as “_ts”, which exists in every document you create.  Basically it is a UNIX epoch timestamp. This field is updated everytime when the document is modified. [Ref: Picture1]

image

[Picture1]

Enabling TTL on Cosmos DB Collection:

You can enable TTL on a Cosmos DB collection simply by using Azure Portal –> Cosmos DB collection setting for existing or during creation of  a new collection)

TTL value needs to be set in seconds – if you need 90 days => 60 sec * 60 min * 24 hour * 90 days = 7776000 seconds

image

[Picture2]

Below is a one of the reference architecture in which Cosmos DB – TTL would be essentially useful and viable to any Iot business case:

image

[Picture3]

Hope that was helpful to get some understanding. For more references visit:  Cosmos DB Documentation

Azure in Germany–a complete EU cloud computing solution

May 18, 2017 .NET, Analytics, AppFabric, Azure, Azure in Germany, Azure IoT Suite, Cloud Computing, Cloud Services, Cloud Strategy, Cognitive Services, Computing, Data Analytics, Data Governance, Data Hubs, Data Warehouse, Emerging Technologies, Event Hubs, IaaS, Intelligent Edge, Internet of Things, IoT, IoT Central, IoT Hub, Machine Learning(ML), Media Services, Media Services & CDN, Messaging, Microsoft, Mobile Services, PaaS, SaaS, SQL Azure, Storage, Backup & Recovery, Stream Analytics, Virtual Machines, Windowz Azure No comments

With my earlier article Azure in China, it came in to my interest to look for any other country/region specific independent cloud data center requirements.  I came across Azure for US Govt(Similar to Amazon Govt Cloud) instance and Azure Germany data center.  For this article context I will be covering only Azure in Germany.

What is Azure Germany?

Just like regional regulatory requirements in China, Germany also wanted a completely locally owned/managed Azure Data Center for EU/EFTA/UK requirements. This is also to ensure stricter access control and data access policy measurements. This  approach is to enable organizations doing business in EU/EFTA and UK can better harness the power of cloud computing.

  • All customer data and related applications and hardware reside in Germany
  • Geo-replication between datacenters in Germany to support  business continuity
  • Highly secured datacenters provide 24×7 monitoring
  • It meets all Public sector or restricted industry requirements
  • Follows all Compliance requirements for EU/EFTA and UK.
  • Lower cost, locally accessible  within your business locations in Germany/EU.

“ Azure Germany is an isolated Azure instance in Germany, independent from other public clouds.”

Who controls it?

An independent data trustee controls access to all customer data in the Azure Germany datacenters. T-Systems International GmbH, a subsidiary of Deutsche Telekom and an experienced, well-respected IT provider incorporated in Germany, serves as trustee, protecting disclosure of data to third parties except as the customer directs or as required by German law.

** Even Microsoft does not have access to customer data or the datacenters without approval from and supervision by the German data trustee.

What Compliance?

Azure Germany has an ongoing commitment to maintaining the strictest data protection measures, so organizations can store and manage customer data in compliance with applicable German laws and regulations, as well as key international standards. Additional compliance standards and controls that address the unique role of the German data trustee will be audited over time. Refer to: Microsoft Trust Center compliance.

[Source : Microsoft Azure]

Useful Links:

Azure in China

May 17, 2017 .NET, AppFabric, Azure, Azure In China, Azure IoT Suite, Cloud Computing, Cloud Services, IaaS, Intelligent Cloud, Intelligent Edge, IoT Hub, Media Services, Microsoft, Mobile Services, PaaS, SQL Azure, SQL Server, VisualStudio, Windowz Azure 1 comment

Microsoft Azure presence in China is always a question when there is a need for any customer to deploy azure applications specifically for Chinese Regional customers.

Recently I had an interaction with a Microsoft Certified Trainer, who carelessly said Azure only uses Chinese partner based environment is only for serving CDN (Content Delivery Network) needs, partial knowledge is difficult to convince. Being a Certified Trainer, he should have known these. As I recollect my earlier experience with azure, I was aware that Microsoft has partnered with 21Vianet , to setup a Chinese instance of Azure with most possible IaaS, PaaS and SaaS offerings are available to Chinese customers. At the same time data would reside within Chinese governments data privacy and storage regulations.

Similarly other competitor cloud providers such as IBM also partnered with 21Vianet to setup their BlueMix instances in China.

Microsoft Azure in China

General availability of Azure in China through 21vianet partnership has been announced in March 26th 2014. These Azure Services in Chinese region will be operated and sold by 21vianet.

Quoting to Doug Hauger GM- National Cloud Programs, Microsoft

Windows Azure, operated by 21Vianet, is based on the same technology as Microsoft’s global Windows Azure service and offers a reliable, flexible and value-based service to bring Microsoft’s latest cloud technologies and benefits to Chinese customers and propel forward technological development in China as a whole.

Why Partnership?

As per Chinese Government(PRC) Cyber Security Law and Governance policies there is a clear restriction for a foreign entrant to obtain license and provide services to Chinese customers.  All the data relating to Chinese customers should reside within Chinese Data Centres, and foreign companies would have to pass on the operational ownership to a Chinese company in order to operate within PRC(Peoples Republic of China).

In Additional to that on November’2016 new amendments attached to new PRC Cyber Security Law (effective 1st June 2017).

As per DLA PIPER

The new law will come into force on 1 June 2017 and has significant implications for the data privacy and cybersecurity practices of both Chinese companies and international organisations doing business in China.

The new PRC Cybersecurity Law intends to combat online fraud and protect China against Internet security risks. In short, it imposes new security and data protection obligations on “network operators”; puts restrictions on transfers of data outside China by “key information infrastructure operators”; and introduces new restrictions on critical network and cybersecurity products.  [Read more from Source]

Where is Azure in China?

You can access Azure in China instance through a separate portal https://www.azure.cn. Azure Portal is available at https://portal.azure.cn/. Pricing information is available on http://www.azure.cn/pricing/pia/  (click on English to get an English version).

Below are two data centre regions Azure is available in China.

image

Services Offered in Azure China Instances:

Not all all services of Azure including Mobile Services, Analytics and IoT Suite are available for Chinese regions. I will try to cover a specific list of services that are available for China region.Azure China is a few versions behind regular Azure and not all services are supported. Some of the services supported are:

  • Compute – Virtual Machines (For IaaS Services)
  • Compute – Cloud Service (For PaaS Services)
  • Compute – Web Site
  • Data Services – Storage (For Blobs, tables and Queues)
  • Data Services – SQL Database
  • Data Service – HDInsight
  • SQL Database – SQL Import/ExportNetworks – Virtual Network (To communicate between cloud services via Local networks)
  • Service Bus
  • Active Directory
  • Access Control Service – which has its own UI and namespace
  • Caching – only dedicated cache is supported
  • Mobile Service
  • Media Service
  • CDN
  • Traffic Manager
  • Azure IoT Suite
  • Notification Center
  • Event Center
  • Service Fabric
  • Batch Service
  • Flow Analytics, Power BI
  • Document DB, SQL Database, SQL Data Warehouse, MySQL
  • and many more

image

A list of supported and not supported services can be found at (This is in Chinese, and it is a working document, use google translate to take you through all steps necessary for you to understand services offered in Azure China) http://www.windowsazure.cn/documentation/articles/developerdifferences/#readyonwacn

This also includes all the endpoints that are required for developers to integrate/develop applications targeting China region.

Hope this information was helpful to you. I would assume it would be helpful for some who are specifically looking forward for providing technology solutions for Chinese customers through Azure Cloud.

Sources:

Introducing Azure IoT Edge

May 13, 2017 .NET, Analytics, Artificial Intelligence(AI), Augmented Reality, Azure, Azure IoT Suite, Cloud Computing, Data Analytics, Edge Analytics, Embedded, Emerging Technologies, Event Hubs, Industrial Automation, Intelligent Cloud, Intelligent Edge, IoT, IoT Edge, IoT Hub, Linux, Mac OSX, Machine Learning(ML), Microsoft, Robotics, Self Driven Cars, Stream Analytics, Windows, Windowz Azure No comments

During Build! 2017 Microsoft has announced the availability of Azure IoT Edge, which would bring in some of the cloud capabilities to edge devices/networks within your Enterprise. This would enable industrial devices to utilize the capabilities of IoT in Azure within their constrained resources . 

With this Microsoft now makes it easier for developers to move some of their computing needs to these devices.  Edge devices are mostly having small foot print based to high end machines within your company network.

The essential capabilities to be supported by Azure IoT edge  include:

  • Perform Edge Analytics (a cut down version of Azure Stream Analytics)- Instead of doing analytics in cloud developer/implementer can move the basic cloud data processing and analytical capabilities to Edge Device. Run your machine learning algorithms in Edge device and take predictive analytics steps.
  • Perform Artificial Intelligence processing at edge device itself. Availability of Microsoft Cognitive Service on edge device would bring in whole lot of automation capabilities. Imagine Alexa/Siri working without internet connection, it should be able to provide you reminders etc.
  • Perform RealTime Decision making locally based on predefined rules.
  • Reduce bandwidth costs
  • Connect to other Edge devices and legacy devices within the constrained/corporate network.
  • Deploy IoT solutions to Edge Device from Cloud and provide updates as needed.
  • Operate offline without the need of real-time internet connectivity or intermittent connectivity. Doesn’t have to rely on Cloud to provide commands for processing, can do offline data capture and processing of information from other devices connected and take decisions without the need to rely on a connected cloud service.

Azure IoT Edge enables seamless deployment of cloud services such as:

Along with sharing the image represents Azure’s Enterprise Digital Vision, we will discuss about the same in later sessions:

Digital-Enterprise-Vision_png

Getting Started & More information:

IoT Central–Microsoft’s SaaS solution for IoT

April 25, 2017 AMQP, Analytics, Azure, Azure IoT Suite, Cloud Computing, Cloud Services, Cloud to Device, Communication Protocols, Connected, Connectivity, Device to Cloud, Emerging Technologies, HTTP 1.1, Identity of Things (IDoT), Intelligent Cloud, Intelligent Edge, Internet of Things, IoT, IoT Central, IoT Devices, IoT Edge, IoT Hub, IoT Privacy, IoT Security, Machines, MQTT, PaaS, SaaS, Stream Analytics No comments

Microsoft has today released their IoT SaaS offering for customers and partners called as “Microsoft IoT Central”.  IoT Central enables powerful IoT scenarios without requiring cloud solution expertise and also simplifies the development process and makes customers to make quick time to market solutions, making digital transformation more accessible to everyone without overhead of implement solutions end to end.

As per Microsoft :

“IoT Central provides an easier way to create connected products that propel digital business. Take the complexity out of the Internet of Things (IoT) with a true, end-to-end IoT software as a service (SaaS) solution in the cloud that helps you build, use, and maintain smart products.”

Benefits of IoT Central:

  • Proven platform and technology with enterprise grade security.
  • Reduced complexities of setting up and maintaining IoT infrastructure and solutions.
  • Building smart connected products with lesser cost  and lesser overhead would ensure higher customer satisfaction.
  • Quickly adapt to changing environments.

For those would need control on implementing end to end can still choose the PaaS solution Azure IoT Suite.

Below is a picture from @JanakiramMSV’s article from forbes.com, to help you have a high level look at all the IoT offerings from Microsoft.

az-iot

Sources:

Azure IoT Hub Device Management–Released to Public

November 17, 2016 .NET, Azure, Azure IoT Suite, Cloud Services, Cloud to Device, Connectivity, Device Shadow, Device to Cloud, Device Twin, Internet Appliance, Internet of Things, IoT, IoT Hub, machine-to-machine (M2M), Microsoft, Tech-Trends No comments ,

Today Microsoft has announced general availability of Azure IoT Hub Device Management. With this release Azure IoT Hub subscribers/customers will be able to get access to following features and functionalities:

  • Device twin. Use a digital representation of your physical devices to synchronize device conditions and operator configuration between the cloud and device.
  • Direct methods. Apply a direct, performant action on a connected device through the cloud.
  • Jobs. Broadcast and schedule device twin changes and methods to scale management operations across millions of devices.
  • Queries. Create real-time, dynamic reports across device twins and jobs to attest status and health for entire device collections, whether your devices are online or offline.

Useful References: