Azure Static Web Apps Login Redirects: How to Stop It

Overview

Integrating custom Microsoft Entra ID (Azure AD) authentication with an Azure Static Web App (SWA) on the Standard tier should be straightforward, but subtle misconfigurations can lead to a frustrating infinite redirect loop.

Typically, you log in, complete MFA, and watch the browser bounce repeatedly between Entra ID and your application until Microsoft halts the flow with Error 50074 (“We couldn’t sign you in”). When you check /.auth/me, you find that clientPrincipal is null.

Azure Static Web Apps Login Redirects: How to Stop It

Here is a breakdown of why this happens and the exact checklist to fix it.

---

The Problem: Why Does the App Loop?

The infinite loop occurs when the interactive user login succeeds in Entra ID, but SWA’s backend proxy fails the subsequent token exchange.

When the token exchange fails:

• 1. SWA cannot establish the StaticWebAppsAuthCookie.
• 2. The user lands back on the app unauthenticated (clientPrincipal: null).
• 3. SWA evaluates the catch-all route guard (/*), issues a 401, and responseOverrides immediately redirects the user back to /.auth/login/aad.
• 4. The cycle repeats until Entra ID detects the loop and blocks access.

Two common triggers for this failure are explicitly passing OAuth parameters (like response_type=code or scope) inside login.loginParameters—which breaks SWA’s built-in token negotiation—and using internal rewrite rules instead of explicit redirect routes.

---

What to Check

• SWA Hosting Plan: Ensure your Azure Static Web App is running on the Standard SKU. Custom OpenID Connect / Entra ID identity providers are ignored on the Free tier.
• Platform Type: Configured strictly as Web (not Single-page application) under App registrations > Authentication.
• Redirect URI: Registered under the Web platform as https://<your-domain>/.auth/login/aad/callback.
• ID Tokens Enabled: In Entra ID > Authentication > Platform configurations > Web > Implicit grant and hybrid flows, ensure the checkbox for ID tokens (used for implicit and hybrid flows) is enabled.
• OpenID Issuer Tenant ID: The openIdIssuer URL must use your actual Directory (Tenant) ID (not an Azure Subscription ID), ending with /v2.0 and no trailing slash.
• Client ID Setting: The Application (Client) ID matches the exact environment variable name referenced in clientIdSettingName.
• Client Secret Value: The SWA configuration setting referenced by clientSecretSettingName must hold the Secret Value (the plaintext string generated on creation), not the Secret ID GUID.
• No loginParameters Overrides: Omit login.loginParameters entirely from your JSON config. Do not manually pass response_type or scope, as SWA handles Authorization Code negotiation out of the box.
• Route Actions: Set the /login route to an explicit redirect with post_login_redirect_uri=/ rather than an internal rewrite to /.auth/login/aad.
• Clean System Endpoints: Do not define custom route rules in routes for SWA system paths like /.auth/login/aad/callback or /.auth/complete.

---

Working staticwebapp.config.json Baseline

Use this base configuration to validate that the login process works properly. You can then make additional changes for your configuration.

{
  "auth": {
    "identityProviders": {
      "azureActiveDirectory": {
        "registration": {
          "openIdIssuer": "https://login.microsoftonline.com/<TENANT_ID>/v2.0",
          "clientIdSettingName": "AZURE_CLIENT_ID",
          "clientSecretSettingName": "AZURE_CLIENT_SECRET"
        }
      }
    }
  },
  "routes": [
    {
      "route": "/login",
      "redirect": "/.auth/login/aad?post_login_redirect_uri=/",
      "statusCode": 302
    },
    {
      "route": "/logout",
      "redirect": "/.auth/logout?post_logout_redirect_uri=/",
      "statusCode": 302
    },
    {
      "route": "/*",
      "allowedRoles": ["authenticated"]
    }
  ],
  "responseOverrides": {
    "401": {
      "redirect": "/.auth/login/aad?post_login_redirect_uri=/",
      "statusCode": 302
    }
  }
}

---

🌟 Let’s Connect & Build Together

Thanks for reading! 😊 If you enjoyed these resources, let’s stay in touch! I share deep-dives into AI/ML patterns and host community events here:

• GDG Broward: Join our local dev community for meetups and workshops.
• Global AI Events: Join Global AI Events.
• LinkedIn: Let’s connect professionally! I share insights on engineering.
• GitHub: Follow my open-source journey and star the repos you find useful.
• YouTube: Watch step-by-step tutorials on the projects listed above.
• BlueSky / X / Twitter: Daily tech updates and quick engineering tips.

👉 Originally published at ozkary.com

Overview Integrating custom Microsoft Entra ID (Azure AD) authentication with an Azure Static Web App (SWA) on the Standard tier should be ...

Read Article →

SaaS Fundamentals: A Primer's Guide for Success

Overview

Have you ever encountered a problem, big or small, and thought, 'Wouldn't it be great if there was software to solve this?' You envision a tool, a solution, something that would make life easier or a process more efficient. That spark of an idea, that 'what if' moment, is where innovation begins. But what if you could do more than just imagine it? What if you could take that spark and transform it into a real, thriving application? With the SaaS approach, that's not just a dream – it's an achievable reality. This presentation will show you the fundamental steps to take your software idea from a simple thought to a successful online service.

#Innovation Series

YouTube Video

Video Agenda

• Introduction to SaaS
• Planning for SaaS Success
• Technical Approach & MVP
• Automation, Security, and Data
• Rollout and Market Presence
• Continuous Retention and Support
• How Do I Get Started?

Presentation

Introduction to Software as a Service (SaaS)

SaaS is a cloud-based software delivery model where users pay monthly or yearly to use it. It does not require the need to install anything or buy licenses.

Features:

• Cloud hosted.
• Monthly or yearly commitment.
• Reduced cost.
• Low risk investment for consumers.
• Ongoing product updates that are seamless to the users.

👉 *Key takeaway: SaaS offers a convenient and cost-effective way to access and use software.

Planning for SaaS Success

A SaaS concept should be about ideas that solve real-world problems and are possible to deliver without breaking the bank.

Concepts:

• Problem Identification: Pinpoint a real-world problem your SaaS will solve. Validate user pain points.
• Market Research: Assess market demand and analyze competitors to understand the landscape.
• Business Models: Define pricing, subscription tiers, and payment options for revenue generation.
• Financial Planning: Create a budget for development, tech, marketing, and support costs.
• Technology Budget: Estimate costs for your tech stack, tools, and third-party services.
• Marketing & Support Costs: Include marketing and customer support expenses in your financial plan.

👉 Key takeaway: Solid planning and an MVP are crucial for early validation.

Technical Approach & Minimum Viable Product (MVP)

The technical foundation of your SaaS is built during the MVP phase. Selecting the right technologies and defining the scope of your initial release are critical for efficient development and early user validation.

Concepts:

• MVP Core Features: Define the essential features that address the core problem your SaaS solves.
• Iterate with Feedback: Plan for continuous iteration based on user feedback collected during the MVP phase.
• Tech Stack: Choose the development tools and platforms that best suit your project's needs.
• Code/No-Code: Evaluate if no-code solutions can efficiently deliver the MVP's core functionality.
• Cloud Provider: Select a reliable cloud hosting provider (Microsoft Azure, Google Cloud, AWS).
• Hosting Options: Consider various hosting options based on scalability and performance requirements.

👉 Key takeaway: A focused MVP and a well-chosen tech stack accelerate early validation and development.

Operational Foundation: Automation, Security, and Data

A robust SaaS solution requires a strong operational foundation. This ensures efficient updates, seamless tenant creation, and secure cloud deployments.

Concepts:

• Build Automation: Implement automation for deployment, updates, and infrastructure management to streamline operations.
• Security: Establish a secure and scalable authentication and authorization system using federated identity management. Importance of data security and user privacy.
• Data Management: Define data storage, backup, and recovery strategies to ensure data integrity and availability.
• Ability to Scale: Design the system to handle increasing user loads and data volumes.

👉 Key takeaway: Use a third-party cloud service that provide those features

Launching Your SaaS: Marketing & Deployment

Launching your SaaS involves more than just coding; it's about building a strong market presence. A well-crafted marketing and product site, coupled with strategic deployment, will drive user adoption.

• Marketing Site:
  • Creating a compelling marketing website to attract users.
  • SEO and content marketing strategies.
  • Use a CRM to track leads.
• Product Site:
  • Creating a user-friendly product website with documentation and support resources.
• Cloud Hosting:
  • Choosing a reliable cloud hosting provider (e.g., AWS, Azure, Google Cloud).
  • Scalability and reliability considerations.
• Marketplace:
  • Exploring opportunities for listing your SaaS on marketplaces (e.g., Azure Marketplace, AWS Marketplace).
  • Benefits of marketplace distribution.

👉 Key takeaway: A strong online presence is essential for attracting and retaining users.

Building Long-Term Success: Retention & Support

The journey doesn't end after launch. Retaining users and providing exceptional support are critical for long-term SaaS success.

• Continuous Retention:
  • Implementing feedback mechanisms to gather user insights.
  • Regularly updating and improving the SaaS product.
  • Building a community around your SaaS.
• Support:
  • Providing excellent customer support through various channels (e.g., chat, email, knowledge base).
  • Proactive support and troubleshooting.
  • Creating a knowledge base.

👉 Key takeaway: Continuous improvement and excellent support are crucial for long-term success.

How Do I Get Started?

To begin your SaaS journey, it's crucial to follow the fundamental concepts we've discussed. These principles will guide you through the process of building a successful and sustainable SaaS solution.

Approach:

• Find a Viable, Real Use Case:
  • Identify a practical problem that can be solved with a SaaS solution, ensuring it's feasible and cost-effective.
• Scope the MVP and Define the Technical Approach:
  • Determine the core features for your Minimum Viable Product (MVP) and select the appropriate technology stack and cloud provider.
• Build a Robust Operational Foundation:
  • Implement automation, establish strong security measures, and create a scalable data management strategy.
• Launch with a Strong Market Presence:
  • Develop a compelling marketing site and a user-friendly product site, and explore marketplace opportunities.
• Focus on Continuous Retention and Support:
  • Prioritize user feedback, deliver ongoing updates, and provide excellent customer support to build long-term success.
• Embrace Iteration for Continuous Growth:
  • Build a culture of continuous improvement, regularly iterating on your product and processes based on user feedback and market trends.

👉 Key takeaway: By following to these fundamental concepts, you can lay a solid foundation for your SaaS journey.

Thanks for reading! 😊 If you enjoyed this post and would like to stay updated with our latest content, don’t forget to follow us. Join our community and be the first to know about new articles, exclusive insights, and more!

Leave comments on this post or contact me at:

• Twitter @ozkary
• BlueSky https://bsky.app/profile/ozkary.bsky.social

👍 Originally published by ozkary.com

Overview Have you ever encountered a problem, big or small, and thought, 'Wouldn't it be great if there was software to solve this?...

Read Article →

Data Engineering Process Fundamentals - Discovery

Introduction

In this series of Data Engineering Process Fundamentals, we explore the Data Engineering Process (DEP) with key concepts, principles and relevant technologies, and explain how they are being used to help us deliver solutions. The first step, and important to never skip, in this process is the Discovery step.

During the discovery step of a Data Engineering Process, we look to identify and clearly document a problem statement, which helps us have an understanding of what we are trying to solve. We also look at our analytical approach to make observations about at the data, its structure and source. This leads us into defining the requirements for the project, so we can define the scope, design and architecture of the solution.

Problem Statement

A Problem Statement is a description of what it is that we are trying to solve. As part of the problem statement, we should provide some background or context on how the data is processed or collected. We should also provide a specific description of what the data engineering process is looking to solve by taking a specific approach to integrate the data. Finally, the objective and goals should also be described with information about how this data will be made available for analytical purposes.

Analytical Approach

The Analytical Approach is a systematic method to observe the data and arrive to insights from it. It involves the use of different techniques, tools and frameworks to make sense of the data in order to arrive to conclusions and actionable requirements.

Dataset Criteria

A Dataset Criteria technique refers to the set of characteristics used to evaluate the data, so we can determine the quality and accuracy of it.

In the data collection process, we should identify the various sources that can provide us with accurate and complete information. Data cleaning and preprocessing needs to be done to identify and eliminate missing values, invalid data and outliers that can skew the information. In addition, we should understand how this data is available for the ongoing integration. Some integrations may require a batch process integration at a scheduled interval. Others may require a real-time integration and/or a combination of batch and real-time processing.

Exploratory Data Analysis

We should conduct exploratory data analysis to understand the structure, patterns and characteristics of the data. We need to make observations about the data, identify the valuable fields, create statistical summaries, and run some data profiling to identify trends, metrics and correlations that are relevant to the main objective of the project.

Tools and Framework

Depending on the size and budget of the organization, the solution can be built with lots of coding and integration, or instead a low-code turn-key solution that provides enterprise quality resources could be used instead. Regardless of the approach, a programming language like Python is a popular programming language for data science and engineers, and it is always applicable. The Python Pandas library is great for data manipulation and analysis. Jupyter notes with Python scripts is great for experiments and discovery.

To run our Python scripts and Jupyter notebooks, we can use Visual Studio Code (VSCode), which is cross-platform Integrated Development Environment (IDE) tool. This tool also enables the integration with source control and deployments platforms like GitHub, so we can maintain version control and automate the deployment and test of our code changes.

To orchestrate the pipelines, we often use a workflow framework like Apache Airflow, Prefect. To host the data, we use data lakes (blob storage) and a relational data warehouse. For data modeling, incremental data and continuous test and data ingestion, Apache Spark or gbt cloud are used.

For the final data analysis and visualization, we could use tools like Looker, PowerBI and Tableau. These are tools that can connect to a data warehouse and consume the data models, so they can visualize in ways that enable stakeholders to make decisions based on the story provided by the data.

Requirements

Requirements refer to the needs, capabilities and constraints that are needed to deliver a data engineering solution. They should outline the project deliverables that are required to meet the main objectives. The requirements should include data related areas like:

• Sources and integration
• Modeling and transformation
• Quality and validation
• Storage and infrastructure
• Processing and Analytics
• Governance and Security
• Scalability and performance
• Monitoring

Design

A data engineering design is the actual plan to build the technical solution. It includes the system architecture, data integration, flow and pipeline orchestration, the data storage platform, transformation and management, data processing and analytics tooling. This is the area where we need to clearly define the different technologies that should be used for each area.

System Architecture

The system architecture is a high-level design of the solution, its components and how they integrate with each other. This often includes the data sources, data ingestion resources, workflow and data orchestration resources and frameworks, storage resources, data services for data transformation and continuous data ingestion and validation, and data analysis and visualization tooling.

Data Pipelines

A data pipeline refers to a series of connected tasks that handles the extract, transform and load (ETL) as well as the extract, load and transform (ELT) operations and integration from a source to a target storage like a data lake or data warehouse.

The use of ETL or ELT depends on the design. For some solutions, a flow task may transform the data prior to loading it into storage. This approach tends to increase the amount of python code and hardware resources used by the hosting environment. For the ELT process, the transformation may be done using SQL code and the data warehouse resources, which often tend to perform great for big data scenarios.

Data Orchestration

Data orchestration refers to the automation, management and coordination of the data pipeline tasks. It involves the scheduling, workflows, monitoring and recovery of those tasks. The orchestration ensures the execution of those tasks, and it takes care of error handling, retry and the alerting of problems in the pipeline.

Summary

The data engineering discovery process involves defining the problem statement, gathering requirements, and determining the scope of work. It also includes a data analysis exercise utilizing Python and Jupyter Notebooks ,or other tools, to extract valuable insights from the data. These steps collectively lay the foundation for successful data engineering endeavors.

Exercise - Hands-on Use Case

Since we now understand the discovery step, we should be able to put that into practice. Let’s move on to a hands-on use case and see how we apply those concepts.

👉 Data Engineering Process Fundamentals - Discovery Exercise

Thanks for reading.

Send question or comment at Twitter @ozkary

👍 Originally published by ozkary.com

Introduction In this series of Data Engineering Process Fundamentals, we explore the Data Engineering Process (DEP) with key concepts, prin...

Read Article →

Data Engineering Process Fundamentals

Introduction

Data Engineering is changing constantly. From cloud data platforms and pipeline automation to data streaming and visualizations tools, new innovations are impacting that way we build today’s data and analytical solutions.

In this series of Data Engineering Process Fundamentals, we explore the Data Engineering Process (DEP) with key concepts, principles and relevant technologies, and explain how they are being used to help us deliver the solution. We discuss concepts and take on a real use case where we execute an end-to-end process from downloading data to visualizing the results.

The end-goal of this series is to take us thru a process in which we deliver an architecture, which facilitates the ongoing analysis of big data via analytical and visualization tools. In the following images, we can get a preview of what we will be delivering as we execute each step of the process.

Data Engineering - Architecture

Data Engineering - Analysis Results

Data Engineering Process

A Data Engineering Process follows a series of steps that should be executed to properly understand the problem statement, scope of work, design and architecture that should be used to create the solution. Some of these steps include the following:

👉 Join this newsletter to receive updates Sign up here

• Discovery
  • Problem Statement
  • Data Analysis
  • Define the Requirements and Scope of Work
  • Discovery Exercise
• Design and Planning
  • Design Approach
  • System Architecture
  • Cloud Engineering and Automation
  • Design Exercise

• Data Orchestration and Operations
  • Pipeline Orchestration
    • Batch Processing
  • Workflow Automation
  • Deployment, Schedules and Monitoring
• Data Warehouse and Modeling
  • Data modeling
  • Data Warehouse Design
  • Continuous Integration
• Data Analysis and Visualization
  • Analyze the data
  • Visualization Concepts
  • Create a Dashboard
    • Provide answers to the problem statement
• Streaming Data
  • Data Warehouse Integration
  • Real-time dashboard

Concepts

What is Data Engineering?

Data Engineering is the practice of designing and building solutions by integrating, transforming and consolidating various data sources into a centralized and structured system, Data Warehouse, at scale, so the data becomes available for building analytics solutions.

What is a Data Engineering Process?

A Data Engineering Process (DEP) is the sequence of steps that engineers should follow in order to build a testable, robust and scalable solution. This process starts really early on with a problem statement to understand what the team is trying to solve. It is then followed with data analysis and requirements discovery, which leads to a design and architecture approach, in which the different applicable technologies are identified.

Operational and Analytical data

Operational data is often generated by applications, and it is stored in transactional databases like SQL Server, CosmosDB, Firebase and others. This is the data that is created after an application saves a user transaction like contact information, a purchase or other activities that are available from the application. This system is not design to support Big Data query scenarios, so the reporting system should not be overloading its resources with large queries.

Analytical data is the transaction data that has been processed and optimized for analytical and visualization purposes. This data is often processed via Data Lakes and stored on Data Warehouse.

Data Pipelines and Orchestration

Data Pipelines are used to orchestrate and automate workflows to move and process the transactional into Data Lakes and Data Warehouse. The pipelines execute repeatable Extract Transform and Load (ETL) or Extract Load and Transform (ELT) processes that can be triggered by a schedule or a data event.

Data Lakes

A Data Lake is an optimized storage system for Big Data scenarios. The primary function is to store the data in its raw format without any transformation. This can include structure data like CSV files, unstructured data like JSON and XML documents, or column-base data like parquet files.

Data Warehouse

A Data Warehouse is a centralized storage system that stores integrated data from multiple sources. This system stores historical data in relational tables with an optimized schema, which enables the data analysis process. This system can also integrate external resources like CSV and parquet files that are stored on Data Lakes as external tables. The system is designed to host and serve Big Data scenarios. It is not meant to be used as a transactional system.

Data Batch Processing

Batch Processing is a method often used to run high-volume, repetitive data jobs. It is usually scheduled during certain time windows that do not impact the application operations, as these processes are often used to export the data from transactional systems. A batch job is an automated software task that may include one or more workflows. These workflows can often run without supervision, and they are monitored by other tools to ensure that the process is not failing.

Streaming Data

Streaming Data is a data source that sends messages with small content but with high volume of messages in real-time. This data often comes from Internet-of-things (IoT) devices, manufacturing equipment or social media sources, often producing a high volume of information per second. This information is often captured in aggregated time windows and then store in a Data Warehouse, so it can be combined with other analytical data. It can also be sent to monitoring and/or real-time systems to show the current system KPI or any type of variance in the system.

Next Step

👉 Data Engineering Process Fundamentals - Discovery

Thanks for reading.

Send question or comment at Twitter @ozkary

👍 Originally published by ozkary.com

Introduction Data Engineering is changing constantly. From cloud data platforms and pipeline automation to data streaming and visualization...

Read Article →

Use Remote Dev Container with GitHub Codespaces

As Software Engineers, we usually work with multiple projects in parallel. This forces us to configure our work stations with multiple software development tools, which eventually leaves our workstation performing poorly. To overcome this problem, we often use virtual machine (VM) instances that run in our workstations or a cloud provider like Azure. Setting up those VMs also introduces some overhead into our software development process. As engineers, we want to be able to accelerate this process by using a remote development environment provider like GitHub Codespaces.

What is GitHub Codespaces?

GitHub Codespaces is a cloud hosted development environment that is associated to a GitHub repository. Each environment or Dev Container is cloud hosted in a Docker container with the core dependencies that are required for that project. The container is hosted on a VM running Ubuntu Linux. The hardware is also configurable. It starts with a minimum of 2 cores, 8 GB of RAMs and 32 GB of storage, which should be a good foundation to run small projects. In addition, the hardware resources can be increased up to 32 cores, 64 GB RAM and 128 GB of storage which matches a very good workstation configuration.

👍 There are monthly quotas for using the remote environments of 120 hrs for personal accounts, and 180 hrs for the PRO account.

How to use Codespaces?

Codespaces leverages the Secure Shell (SSH) protocol, which provides a secure channel between client and server.  This protocol is used to provide remote access to resources like VMs that are hosted on cloud platforms. This protocol makes it possible for browsers and IDE tools like VS Code to connect remotely and manage the projects.

When using the browser, the VS Code Browser IDE is loaded. We can also use a local installation of VS Code or any IDE that support SSH. The development process works the same way as if running locally with the exception that the files are hosted remotely, and we can also use a terminal window to execute build commands within the VM space.

How to start a project with GitHub Codespaces

We can start a Codespaces environment right from GitHub. Open a repo on GitHub and click on the Code tab and then click the Code button from the toolbar. This opens up the options to create a new Codespaces, connect to an existent one, or even configure your Codespaces resources, more on this later. 

👍 You can use this repo if you do not have one https://github.com/ozkary/Data-Engineering-Bootcamp

When you add a new environment, GitHub essentially provisions a VM on Azure. It loads a Docker image with some of the dependencies of your project. For example, if your code is .Net Core or a TypeScript with React project, a Docker image with those dependencies is built and provisioned into the VM.

👍 The Docker images are preconfigured. We can also build a custom image to meet specific requirements.

Once the environment is provisioned, we can open the project using any of the options listed on the image below. I prefer to use my local VS Code instance, as I often have all the tools needed to work on my projects. Once the project is open on VS Code, the project connection is cached, and we only need to open VS Code again to load the remote project. The browser feature is also very useful, so do take it for a spin and see how you like it.

Use a Terminal to Manage the Project

When the project is open remotely, we can run common activities like adding additional dependencies, building and debugging the project. Since the environment is running on Ubuntu, we can open a terminal window on VS Code. This enables us to run the CLI commands that we need in order to manage our project. 

In the case of Web projects, we can run the project remotely using our browser. Even though the project runs remotely on the VM, port forwarding is used for secured remote access, so we can open our local browser and load the app. We can see the forwarded ports for our application on the ports tab of VS Code.

Managing your Codespaces Instance

In some cases, we may see some performance issue on our remote environment. If this is the case, we need to inspect the current instance configuration and if possible upgrade the resources. Since this is an Ubuntu instance, we can use the terminal to run commands like lscpu to check the current configuration like cpu’s and memory. We can also use the Codespaces command toolbar and change the machine type, which provides a quick shortcut to change the machine type or configure the container. 

The Dev Container can also be customized by making changes to the devcontainer.json file. Additional customization can be done by building a custom Docker image to meet specific development environment requirements.

👍 When the Dev Container is changed, the VM requires a re-start, which is done automatically

Summary

By leveraging the use of remote managed development environments, software engineers can save time by not having to work on a development environment configuration, we can instead use GitHub Codespaces to quickly provision Dev Containers that get us up and running in a short time, thus allowing us to focus on our development tasks instead of environment managing tasks.

Thanks for reading.

Send question or comment at Twitter @ozkary

Originally published by ozkary.com

As Software Engineers, we usually work with multiple projects in parallel. This forces us to configure our work stations with multiple softw...

Read Article →

Reduce code complexity by letting an API Gateway handle disparate services and document transformation

Modern Web applications use the microservice architecture for their API service integration. These APIs are often a combination of internal and external systems. When the system is internal, there is better control of the API endpoints and contract payload which the front-end components consume.  When there are external systems, there is no real control as the endpoints and contracts can change with new version updates.

There are also cases when the integration to these external integrations must be done with multiple providers to have some redundancy. Having to integrate with multiple providers, forces the application to manage different endpoints and contracts that have different structure. For these cases, how does the client application know what API endpoint to call? How does it manage the different structure and formats, JSON or XML, on both the request and response contracts? What is the approach when a new external service is introduced? Those are concerning questions that an API Gateway can help manage.

What is an API Gateway?

An API Gateway is an enterprise cloud solution that integrates client applications to back-end services or APIs. It works as a reverse proxy which forwards inbound requests to internal or external services. This approach abstracts the service's endpoint details from the application; therefore, an application only needs to be aware of the gateway endpoint information.  

When dealing with disparate services, an application must deal with the different contracts and formats, JSON, XML, for the request and subsequent response. Having code on the application to manage those contracts, leads to unmanageable and complex transformation code. A gateway provides transformation policies that enables the client application to only send and receive one contract format for each operation. The gateway transformation pipeline processes the request and maps it to the contract schema required by the service. The same takes place with the response, as the payload is also transformed into the schema expected by the client application. This isolates all the transformation process in the gateway and removes that concern from the client.

API Settings

To better understand how an API Gateway can help our apps avoid a direct connection to the services, we should learn about how those services and their operations should be configured.  To help us illustrate this, let’s think of an integration with two disparate providers, as shown on the image below.

The client apps can be using the APIs from either Provider A or B. Both providers are externally located in a different domain, so to manage the endpoint information, the apps are only aware of the gateway base URL.  This means that independently of how many providers we may add to this topology, the clients always connect to the same endpoint.  But wait, this still leaves us with an open question. How is the routing to a specific provider handled?

Operation Routing

Once we have the base URL for the gateway endpoint, we need to specify the routing to the API and specific operation. To set that, we first need to add an API definition to the gateway. The API definition enables us to add an API suffix to the base URL. This suffix is part of the endpoint route information and precedes the operation route information.

An API definition is not complete unless we add the operations or web actions which handle the request/response. An operation defines the resource name, HTTP method and route information that the client application uses to call the API endpoint in the gateway. Each route maps to an operation pipeline which forward requests to the provider’s API endpoint and then sends the response back to the client. In our example, the routing for the operation of Provider A looks as follows:

This image shows us how an API has a prefix as well as operations. Each of the operations is a route entry which completes the operation URL path. This information, plus the base URL, put together handles the routing of a client request to a particular operation pipeline, which runs a series of steps to transform the documents and forward the request to the provider’s operation.

Note: By naming the operations the same within each API, only the API suffix should change. From the application standpoint, this is a configuration update via a push update or a function proxy configuration update.

Operation Pipeline

The operation pipeline is a meta-data driven workflow. It is responsible for managing the mapping of the routing information and execution of the transformation policies for both the request and response. The pipeline has four main steps: Frontend, Inbound, Backend and Outbound.

The Frontend steps handles the Open API specifications JSON document. It defines the hostname, HTTP schemes, and security requirements for the API. It also defines, for each operation, the API route, HTTP method, request parameters or model schema for both the request and response. The models are the JSON contracts that the client application sends and receives.

The Inbound step runs the transformation policies. This includes adding header information, rewrites the URL to change the operation route into the route for the external API. It also handles the transformation of the operation request model into the JSON or XML document for the external API. As an example, this is the step that transform a JSON payload into SOAP by adding the SOAPAction header and SOAP envelope into the request.

The Backend step defines the base URL for the target HTTP endpoint. Each operation route is appended to the backend base URL to send the request to the provider. On this step, security credentials or certificated can be added.

Lastly, the Outbound step, like the Inbound step, handles header and document transformation before the response is sent back to the client application. It transforms the JSON or XML payload into the JSON model defined by the Frontend schema configuration. This also the place to add error handling document standards for the application to handle and log accordingly independently of the provider.

This is an example of a transformation policy which shows an inbound request transformed to SOAP and outbound response transformed to JSON.

Conclusion

In Microservice architecture, a client application can be introduced to disparate APIs which support multiple document structure and different endpoints, as these API services are hosted in different domains. To avoid complex code which deal with multiple document formats and endpoints, an API Gateway can be used instead. This enables us to use meta-data driven pipelines to manage that complexity away from the app. This should enable the development teams to focus on app design and functional programming instead of writing code to manage infrastructure concerns.

Have you faced this challenge before, and if so, what did you do to resolve it? If you use code in your app, what did you learn from that experience?

Send question or comment at Twitter @ozkary

Originally published by ozkary.com

Modern Web applications use the microservice architecture for their API service integration. These APIs are often a combination of intern...

Read Article →

Understand Your Users with App Insights to Improve Adoption

After spending several months of software requirements, scrum and design meetings, hours of implementation and testing efforts, our application is finally deployed, but the work for a sustainable product is not complete. After the app is deployed, it is important to understand the users to improve its adoption, and to understand its users, many questions need to be answered. Some common questions may include:

How do we know about user acceptance? How can we measure the user experience? Are all those different features on the app being used? How can we learn from our users to make app improvements?

To answer those question, we need to collect data that can be used to provide and validate the answers.  The best approach to collect this data is by adding telemetry tracking in our apps. By tracking user activities like page views, clicks and custom events, we can use Application Insights analytical visualization tools to understand the user’s behavior. In this article, we look at some of the analytical tools that are available on Azure Monitor Services, which we can leverage to help us answer the questions listed before and better understand our users. Before we get to understand those tools, let’s first provide an overview of the monitor service that host them.

Application Insights (AppInsights)

AppInsights is part of the Cloud Monitor Services on Azure. It has two main areas that work tightly together. It is a monitor service with statistical analysis tooling, and it is an application framework, with support for languages like JavaScript, C#, Python, that is used to instrument our applications and track telemetry information for our apps.

This service has multiple statistical analysis tool to help digest the telemetry information for many areas of the apps which can include front-end, back-end, APIs. The tools are also grouped on areas of concerns like investigation for performance metrics and problems, monitoring/alert, and application usage.

It is the usage statistical analysis tooling that can help us learn how are users interacting with the application. Some of these tools include the users, sessions, events, cohorts, funnels, and flows tools. Let’s review each one of these tools and see how they can help us understand our users and their behavior.

Users Analysis

The user analysis tool helps us understand details about the users. We can visualize what events and pages/views the users are interested in as well as some user specifics like country or city of origin, operating system in their devices, and browser versions.

Those details can help us make decisions like improving advertisement on some geographical areas to attract those users. Also, they are great indicators to understand the usability based on the device types and browsers being used. This leads to making decision on open issues that are related to mobile device improvement, as an example.

Sessions Analysis

The session analysis tool is very similar to the users’ tool, but it provides session-based information about those users. Keep in mind that one user can have multiple session. These sessions can help us see how users are using the application. The sessions can be visualized by tracking page views and user events. This information is valuable to be able to understand what areas of the application are mostly used. Analysis can be done to understand usability problems in some areas, so they can be improved or simply removed from the app altogether.  

The session information is very valuable for some A/B testing analysis, in which design variants can be introduced to the user experience. This, for example, can reveal that users tend to have a much better user experience with one design over the other.

Events

When we need to understand how users are responding to a particular design or feature on the application, we can use the events tool to see the telemetry information that is collected when the user makes some actions like clicking on tabs, links, dropdowns to change selections. The collected insights from these actions can reveal if a particular feature is relevant to users. This can help determine that perhaps a feature is not needed at all or if a design is too confusing to our users.

Custom events not only track user actions, but they can also be used to track some data and or integration telemetry. For example, we can track events about data variations that comes from an external API. The application may not have any control about the source of the data, but custom events can help us track those variations, which leads into creating some countermeasures to handle them.

Other areas that can be tracked with custom events can include input validation, timeouts from user inactivity, chat or help request from specific areas of the application. All these leads to better insights for the improvement of the overall user experience.

Cohorts

Cohorts enable us to create a specific group of users which have completed a specific goal metric like signing for newsletters, completed a purchase or landed on a particular area of the app. By predefining groups of users with a particular criterion, we can use that group as a filter on the users, sessions, and events tools. This enables us to continuously analyze the same criteria and compare performance metrics.

The advantage of using Cohorts is that we can use custom logs query expressions to select the information we need from the customEvents and pageViews log tables. The customEvents table has all the events that are sent from the app, while the pageViews table contains all the page visit information. Combining these tables with a particular criterion can generate an excellent filter which can enable us to analyze the data in more detail.

Funnels

Most apps have a particular workflow that we would ideally like our users to follow. This however may not always be the case, and users may not follow the steps as expected. Therefore, we require a tool that can help us analyze the data to verify how are the users running each step of the workflow. Funnels represent a set of steps on the application and the percentage of users who follow such steps.

For example, an app workflow may consist of several pages/steps to allow the user to register. Since each page visit is tracked, we can create a funnel and select each page visit as a separate step, thus creating a workflow. The output should indicate to us which step the users are not completing or when they stop or abandon the workflow. This is an indication that a particular step may have some design issues that are causing our users to drop off and not complete the workflow. This information should help identify design improvements on those pages to help guide the user and improve the completion percentage.

User Navigation

Whereas Funnels can help us understand the percentage of users completing certain application steps, it does not provide to us enough information to clearly see where the remaining percentage of users are going.

User Navigation visualization tool provides this insight. The tool provides a page view visualization that can help us track how users are navigating the site. Ideally, the workflow should be a sequential flow, but if we see that users are flowing/going into other areas of the app or stopping at a particular step, we can analyze that information further and make some design changes that can help increase the completion percentage.

To use the tool, an initial event like a page visit can be selected as the starting point, then custom events and other page views can be included to see a flow diagram of the steps before and after the selected target event.

When it comes to user visiting pages, this is by far the most useful tool for us to examine, as this can lead us into better understanding of our application and make design decisions on how to improve the user experience and app navigation which help improve the page visit goals for our apps.

Conclusion

When an application development life cycle starts, several decisions are made based on the information that is available during the design phase of the app. It is very important to validate those design decisions by instrumenting our application, so the necessary telemetry information can be gathered and analyzed post deployment. It is then that we can better understand the app users to help validate the design decisions or make changes to improve both the user experience and adoption of our apps. In addition, this is an ongoing process as user behavior can change over time, so ongoing iteration of this process is required.

Add AppInsights to Single Page Apps

Thanks for reading!

Originally published by ozkary.com

After spending several months of software requirements, scrum and design meetings, hours of implementation and testing efforts, our applica...

Read Article →

Modernize SOAP Web Service with Azure API Management

Simple Object Access Protocol (SOAP) is an XML based protocol standard to build Web Services. It provides a communication channel for data exchange between applications that are built with different technologies. As a result, most companies have invested heavily in building SOAP Web Services.

SOAP, however, has not been the de facto approach for building modern Web application for some time now.  With the creation of new protocols and standards, REST (Representational State Transfer) and JSON (JavaScript Object Notation) are now widely used for new projects, leaving SOAP services mostly on maintenance or support mode only.

SOAP began to lose momentum because of its limitations. The XML message format is a verbose compared to the JSON format. SOAP is also tightly coupled as the client-server communication uses a strict contract definition, WSDL (Web Service Description Language). RESTful APIs in contrast provide an architectural style with loose guidelines and supports several message formats like plain text, XML, JSON, HTML.

Replacing a SOAP Service

The decision to replace a SOAP service with other technologies can be a challenging one. Depending on how complex the project is, it may just be a tremendous investment full of risks for any company. In addition, there are the operation cost to support the new technology and still support clients that cannot migrate to the new API definition.

Some companies may also have the argument that their APIs are doing a great job, and they do not need to be updated with new technology.  For some clients, however, it may be a risk to take on an integration investment with a company that supports, what many perceive as, older technology, or that there are security limitations. Luckily for situations like this, there are alternatives that can meet both sides of the argument.

Modernize a SOAP API

With the growth of cloud technology, there are ways to modernize a legacy API without having to re-write the API which immediately becomes very appealing as this effort saves companies with project management and implementation cost.  For a solution like this, we need a system that can provide a reverse-proxy and document format mapping capabilities.  Azure cloud provides the perfect solution with the API Management service offering.

API Management (APIM) Service

Azure APIM service accelerates this kind of solutions by providing multiple capabilities which enables the managing of APIs by protecting the resources with authentication, authorization, IP Whitelisting, and usage limits. It also provides reverse-proxy capabilities which enable companies to publish REST-based APIs by creating Open API endpoints which act as façade for the legacy systems. It supports the transformation of XML documents into JSON documents and Vice-Versa which makes it the obvious solution for modernizing a legacy SOAP API.

Modernization Process

The process to modernize a legacy SOAP API is much simpler and takes lot of less effort than to take on a rewriting effort. This is mostly due to the metadata reading capabilities of the APIM service. The process starts by importing a WSDL document into the service which reads the metadata to create facades with mapping policies for inbound and outbound stages of the request lifecycle.  The façade is the Open API definition of the RESTful service. The mapping policies enable the mapping and transformation of JSON to XML documents for inbound processing as well as XML to JSON transformation for outbound processing. The Inbound stage handles the JSON payload request which needs to be transformed into XML, so it can be proxied to the legacy SOAP API.  The outbound stage handles the XML response from the legacy SOAP service, and it transforms the response into a JSON document, so the client application can consume it.

Sample Process

Now that there is a good conceptual understanding of the approach, let’s look at a practical example by using a publicly available SOAP API which returns a list of continents by name.  Note that any other SOAP API can be used instead as the process is really the same. Let’s review the SOAP API specifications for this Web service.

Service endpoint:

http://webservices.oorsprong.org/websamples.countryinfo/CountryInfoService.wso

Open or use any tool like Postman that can enable sending API requests to the service. Send a POST request with this header Content-Type: text/xml.  Also add the following for the request body payload.

<?xml version="1.0" encoding="utf-8"?> <soap12:Envelope xmlns:soap12="http://www.w3.org/2003/05/soap-envelope">   <soap12:Body>     <ListOfContinentsByName xmlns="http://www.oorsprong.org/websamples.countryinfo">     </ListOfContinentsByName>   </soap12:Body> </soap12:Envelope>

The expected response should be an XML document that looks as follows:

<?xml version="1.0" encoding="utf-8"?> <soap:Envelope xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/">     <soap:Body>         <m:ListOfContinentsByNameResponse xmlns:m="http://www.oorsprong.org/websamples.countryinfo">             <m:ListOfContinentsByNameResult>                 <m:tContinent>                     <m:sCode>AF</m:sCode>                     <m:sName>Africa</m:sName>                 </m:tContinent>                 <m:tContinent>                     <m:sCode>AN</m:sCode>                     <m:sName>Antarctica</m:sName>                 </m:tContinent>                 <m:tContinent>                     <m:sCode>AS</m:sCode>                     <m:sName>Asia</m:sName>                 </m:tContinent>                 <m:tContinent>                     <m:sCode>EU</m:sCode>                     <m:sName>Europe</m:sName>                 </m:tContinent>                 <m:tContinent>                     <m:sCode>OC</m:sCode>                     <m:sName>Ocenania</m:sName>                 </m:tContinent>                 <m:tContinent>                     <m:sCode>AM</m:sCode>                     <m:sName>The Americas</m:sName>                 </m:tContinent>             </m:ListOfContinentsByNameResult>         </m:ListOfContinentsByNameResponse>     </soap:Body> </soap:Envelope>

On Postman, this configuration with the request and response should look like this:

This example uses XML envelopes for the request and response. To modernize this API, we can import the WSDL definition of the API into Azure APIM. To get the WSDL file, just add the wsdl parameter to the query string and send a GET request. The response should be an XML schema file which describes the API operations and data types.

http://webservices.oorsprong.org/websamples.countryinfo/CountryInfoService.wso?wsdl

Import the WSDL

Note: The following steps should be done on a subscription that has an API Management service configured.

To import the API schema definition into APIM, open Azure Portal, API Management console. Click on the API link to open the existent API definitions. From this view, click on Add API and select the WSDL option. Configure the popup information as shown on the image below. Make sure to use the URL that has the wsdl parameter on the query string. When ready, press create to generate the API definition with policies.

Note: Make sure to use the URL that has the wsdl parameter on the query string

A new API with the name of ContryInfoService should get created. Click on that name to display the operations that are available from this API. Check the settings for this API and make a note of the Base URL. This is the URL that should be used to send request to the new API.

 We are interested in finding the ListOfContinentsByName operation. This is the same API that we use for the Postman validation.  Once selected, review the inbound and outbound policies by clicking on the code icon as shown below:

Policy File

APIM Policy Gist

The policy has an inbound and outbound node. The inbound node uses a liquid template to transform the incoming payload into a SOAP envelope which is required by the SOAP service.  The outbound node also uses a liquid template to transform the XML response into a JSON payload which is then sent back to the calling application. The markup use on the liquid template has many features, and we could no cover all the details here.  For this simple example, we can see how a new JSON document is created by adding the items from the continents collection.

We should now have a JSON API ready, but we are not yet RESTful. The SOAP service requires documents to be send for every request, as a result it uses POST request method for even reading information. A RESTful service should instead be able to get the continent information using a GET request method instead of using POST which should only be used for creating new resources. We should remove that constraint by using APIM to change that configuration as well. Let’s load the operation settings and click on the Frontend configuration. From this configuration, we can change the request method from POST to GET, and even change the URL route to something that aligns with our naming conventions. Also notice that on the inbound policy, we add the set-method policy to indicate that even if the request came in as a GET request, we want to forward this as a POST request which is what the SOAP API expects. The new setting should look as follows:

All should be setup to make requests to the new API. If no extra configuration was done like adding a subscription key, and the APIM resource was not created in a private subnet, we should be able to access the API from the internet. Let’s load our tool of choice and make a request to the new API. Get the Base URL from the setting and append the operation name. That is the new RESTful route. We can now configure a request with the header Content-Type: application/json and set the method to GET. There is no need to add anything on the body as there is no JSON payload to send during a GET operation. This is what the new configuration should look like.

As we can see the results show a list of continents, but instead of showing an XML document it is now showing the data on a JSON document.

Conclusion

Modernizing a legacy SOAP API can be a very expensive and long project. In cases when it is needed to accelerate the solution and minimize the risks, Azure API Management provides a solid approach. This service provides capabilities to create a REST-Base API, policies to handle document transformation and enhances the security of the APIs. All this can be achieved without having to decommission the existent legacy SOAP API and getting a business to provide modern APIs to its customers.

Originally published by ozkary.com

Simple Object Access Protocol (SOAP) is an XML based protocol standard to build Web Services. It provides a communication channel for data ...

Read Article →