Azure Event Grid event schema

 Events consist of a set of five required string properties and a required data object. The properties are common to all events from any publisher. The data object has properties that are specific to each publisher. For system topics, these properties are specific to the resource provider, such as Azure Storage or Azure Event Hubs.

Event sources send events to Azure Event Grid in an array, which can have several event objects. When posting events to an event grid topic, the array can have a total size of up to 1 MB. Each event in the array is limited to 64 KB (General Availability) or 1 MB (preview). If an event or the array is greater than the size limits, you receive the response 413 Payload Too Large.

[

  {

    "topic": string,

    "subject": string,

    "id": string,

    "eventType": string,

    "eventTime": string,

    "data":{

      object-unique-to-each-publisher

    },

    "dataVersion": string,

    "metadataVersion": string

  }

]

Property	Type	Description
topic	string	Full resource path to the event source. This field isn't writeable. Event Grid provides this value.
subject	string	Publisher-defined path to the event subject.
eventType	string	One of the registered event types for this event source.
eventTime	string	The time the event is generated based on the provider's UTC time.
id	string	Unique identifier for the event.
data	object	Event data specific to the resource provider.
dataVersion	string	The schema version of the data object. The publisher defines the schema version.
metadataVersion	string	The schema version of the event metadata. Event Grid defines the schema of the top-level properties. Event Grid provides this value

source: https://docs.microsoft.com/en-us/azure/event-grid/event-schema

Azure CDN Query String & Caching behaviour

Azure Content Delivery Network (CDN) offers two ways to control how your files are cached:

• Caching rules: This article describes how you can use content delivery network (CDN) caching rules to set or modify default cache expiration behavior both globally and with custom conditions, such as a URL path and file extension. Azure CDN provides two types of caching rules:
  • Global caching rules: You can set one global caching rule for each endpoint in your profile, which affects all requests to the endpoint. The global caching rule overrides any HTTP cache-directive headers, if set.
  • Custom caching rules: You can set one or more custom caching rules for each endpoint in your profile. Custom caching rules match specific paths and file extensions, are processed in order, and override the global caching rule, if set.
• Query string caching: You can adjust how the Azure CDN treats caching for requests with query strings. For information, see Control Azure CDN caching behavior with query strings. If the file is not cacheable, the query string caching setting has no effect, based on caching rules and CDN default behaviors.

Caching behavior settings

For global and custom caching rules, you can specify the following Caching behavior settings:

• Bypass cache: Do not cache and ignore origin-provided cache-directive headers.
• Override: Ignore origin-provided cache duration; use the provided cache duration instead. This will not override cache-control: no-cache.
• Set if missing: Honor origin-provided cache-directive headers, if they exist; otherwise, use the provided cache duration.

With Azure Content Delivery Network (CDN), you can control how files are cached for a web request that contains a query string. In a web request with a query string, the query string is that portion of the request that occurs after a question mark (?). A query string can contain one or more key-value pairs, in which the field name and its value are separated by an equals sign (=). Each key-value pair is separated by an ampersand (&). For example, http://www.contoso.com/content.mov?field1=value1&field2=value2. If there is more than one key-value pair in a query string of a request, their order does not matter.

Three query string modes are available:

• Ignore query strings: Default mode. In this mode, the CDN point-of-presence (POP) node passes the query strings from the requestor to the origin server on the first request and caches the asset. All subsequent requests for the asset that are served from the POP ignore the query strings until the cached asset expires.
• Bypass caching for query strings: In this mode, requests with query strings are not cached at the CDN POP node. The POP node retrieves the asset directly from the origin server and passes it to the requestor with each request.
• Cache every unique URL: In this mode, each request with a unique URL, including the query string, is treated as a unique asset with its own cache. For example, the response from the origin server for a request for example.ashx?q=test1 is cached at the POP node and returned for subsequent caches with the same query string. A request for example.ashx?q=test2 is cached as a separate asset with its own time-to-live setting.

source:https://docs.microsoft.com/en-us/azure/cdn/cdn-query-string

Types of filters in Azure Service Bus Topic

Boolean Filters	SQL Filters	Correlation Filter
The TrueFilter and FalseFilter either cause all arriving messages (true) or none of the arriving messages (false) to be selected for the subscription.	A SqlFilter holds a SQL-like conditional expression that is evaluated in the broker against the arriving messages' user-defined properties and system properties. All system properties must be prefixed with sys. in the conditional expression. The SQL-language subset for filter conditions tests for the existence of properties (EXISTS), as well as for null-values (IS NULL), logical NOT/AND/OR, relational operators, simple numeric arithmetic, and simple text pattern matching with LIKE.	A CorrelationFilter holds a set of conditions that are matched against one or more of an arriving message's user and system properties. A common use is to match against the CorrelationIdproperty, but the application can also choose to match against ContentType, Label, MessageId, ReplyTo, ReplyToSessionId, SessionId, To, and any user-defined properties. A match exists when an arriving message's value for a property is equal to the value specified in the correlation filter. For string expressions, the comparison is case-sensitive. When specifying multiple match properties, the filter combines them as a logical AND condition, meaning for the filter to match, all conditions must match.
	Use of SQL Filter rules results in lower overall message throughput at the subscription, topic, and namespace level.	Whenever possible, applications should choose correlation filters over SQL-like filters, since they are much more efficient in processing and therefore have less impact on throughput.
	With SQL filter conditions, you can define an action that can annotate the message by adding, removing, or replacing properties and their values. The action uses a SQL-like expression that loosely leans on the SQL UPDATE statement syntax. The action is performed on the message after it has been matched and before the message is selected into the subscription. The changes to the message properties are private to the message copied into the subscription.

Subscribers can define which messages they want to receive from a topic. These messages are specified in the form of one or more named subscription rules. Each rule consists of a condition that selects particular messages and an action that annotates the selected message. For each matching rule condition, the subscription produces a copy of the message, which may be differently annotated for each matching rule.

Each newly created topic subscription has an initial default subscription rule. If you don't explicitly specify a filter condition for the rule, the applied filter is the true filter that enables all messages to be selected into the subscription.

All filters evaluate message properties. Filters cannot evaluate the message body.

Policies in Azure API Management

In Azure API Management (APIM), policies are a powerful capability of the system that allow the publisher to change the behavior of the API through configuration. Policies are a collection of Statements that are executed sequentially on the request or response of an API. Popular Statements include format conversion from XML to JSON and call rate limiting to restrict the amount of incoming calls from a developer. Many more policies are available out of the box.

Policies are applied inside the gateway which sits between the API consumer and the managed API. The gateway receives all requests and usually forwards them unaltered to the underlying API. However a policy can apply changes to both the inbound request and outbound response.

Policy expressions can be used as attribute values or text values in any of the API Management policies, unless the policy specifies otherwise. Some policies such as the Control flow and Set variable policies are based on policy expressions. For more information, see Advanced policies and Policy expressions.

The policy definition is a simple XML document that describes a sequence of inbound and outbound statements. The XML can be edited directly in the definition window. A list of statements is provided to the right and statements applicable to the current scope are enabled and highlighted.

<policies>
  <inbound>
    <!-- statements to be applied to the request go here -->
  </inbound>
  <backend>
    <!-- statements to be applied before the request is forwarded to 
         the backend service go here -->
  </backend>
  <outbound>
    <!-- statements to be applied to the response go here -->
  </outbound>
  <on-error>
    <!-- statements to be applied if there is an error condition go here -->
  </on-error>
</policies> 

If there is an error during the processing of a request, any remaining steps in the inbound, backend, or outbound sections are skipped and execution jumps to the statements in the on-error section. By placing policy statements in the on-error section you can review the error by using the context.LastError property, inspect and customize the error response using the set-body policy, and configure what happens if an error occurs. There are error codes for built-in steps and for errors that may occur during the processing of policy statements. For more information, see Error handling in API Management policies.

Policies

• Access restriction policies
  • Check HTTP header - Enforces existence and/or value of a HTTP Header.
  • Limit call rate by subscription - Prevents API usage spikes by limiting call rate, on a per subscription basis.
  • Limit call rate by key - Prevents API usage spikes by limiting call rate, on a per key basis.
  • Restrict caller IPs - Filters (allows/denies) calls from specific IP addresses and/or address ranges.
  • Set usage quota by subscription - Allows you to enforce a renewable or lifetime call volume and/or bandwidth quota, on a per subscription basis.
  • Set usage quota by key - Allows you to enforce a renewable or lifetime call volume and/or bandwidth quota, on a per key basis.
  • Validate JWT - Enforces existence and validity of a JWT extracted from either a specified HTTP Header or a specified query parameter.
• Advanced policies
  • Control flow - Conditionally applies policy statements based on the evaluation of Boolean expressions.
  • Forward request - Forwards the request to the backend service.
  • Limit concurrency - Prevents enclosed policies from executing by more than the specified number of requests at a time.
  • Log to Event Hub - Sends messages in the specified format to a message target defined by a Logger entity.
  • Mock response - Aborts pipeline execution and returns a mocked response directly to the caller.
  • Retry - Retries execution of the enclosed policy statements, if and until the condition is met. Execution will repeat at the specified time intervals and up to the specified retry count.
  • Return response - Aborts pipeline execution and returns the specified response directly to the caller.
  • Send one way request - Sends a request to the specified URL without waiting for a response.
  • Send request - Sends a request to the specified URL.
  • Set HTTP proxy - Allows you to route forwarded requests via an HTTP proxy.
  • Set variable - Persist a value in a named context variable for later access.
  • Set request method - Allows you to change the HTTP method for a request.
  • Set status code - Changes the HTTP status code to the specified value.
  • Trace - Adds a string into the API Inspector output.
  • Wait - Waits for enclosed Send request, Get value from cache, or Control flow policies to complete before proceeding.
• Authentication policies
  • Authenticate with Basic - Authenticate with a backend service using Basic authentication.
  • Authenticate with client certificate - Authenticate with a backend service using client certificates.
  • Authenticate with managed identity - Authenticate with a backend service using a managed identity.
• Caching policies
  • Get from cache - Perform cache look up and return a valid cached response when available.
  • Store to cache - Caches response according to the specified cache control configuration.
  • Get value from cache - Retrieve a cached item by key.
  • Store value in cache - Store an item in the cache by key.
  • Remove value from cache - Remove an item in the cache by key.
• Cross domain policies
  • Allow cross-domain calls - Makes the API accessible from Adobe Flash and Microsoft Silverlight browser-based clients.
  • CORS - Adds cross-origin resource sharing (CORS) support to an operation or an API to allow cross-domain calls from browser-based clients.
  • JSONP - Adds JSON with padding (JSONP) support to an operation or an API to allow cross-domain calls from JavaScript browser-based clients.
• Transformation policies
• Convert JSON to XML - Converts request or response body from JSON to XML.
• Convert XML to JSON - Converts request or response body from XML to JSON.
• Find and replace string in body - Finds a request or response substring and replaces it with a different substring.
• Mask URLs in content - Re-writes (masks) links in the response body so that they point to the equivalent link via the gateway.
• Set backend service - Changes the backend service for an incoming request.
• Set body - Sets the message body for incoming and outgoing requests.
• Set HTTP header - Assigns a value to an existing response and/or request header or adds a new response and/or request header.
• Set query string parameter - Adds, replaces value of, or deletes request query string parameter.
• Rewrite URL - Converts a request URL from its public form to the form expected by the web service.
• Transform XML using an XSLT - Applies an XSL transformation to XML in the request or response body.

Source : https://docs.microsoft.com/en-us/azure/api-management/api-management-policies

Offline Data Sync in Azure Mobile Apps

Offline data sync is a client and server SDK feature of Azure Mobile Apps that makes it easy for developers to create apps that are functional without a network connection.

When your app is in offline mode, you can still create and modify data, which are saved to a local store. When the app is back online, it can synchronize local changes with your Azure Mobile App backend. The feature also includes support for detecting conflicts when the same record is changed on both the client and the backend. Conflicts can then be handled either on the server or the client.

Offline sync has several benefits:

• Improve app responsiveness by caching server data locally on the device
• Create robust apps that remain useful when there are network issues
• Allow end users to create and modify data even when there is no network access, supporting scenarios with little or no connectivity
• Sync data across multiple devices and detect conflicts when the same record is modified by two devices
• Limit network use on high-latency or metered networks

What is a sync table?

To access the "/tables" endpoint, the Azure Mobile client SDKs provide interfaces such as IMobileServiceTable(.NET client SDK) or MSTable (iOS client). These APIs connect directly to the Azure Mobile App backend and fail if the client device does not have a network connection.

To support offline use, your app should instead use the sync table APIs, such as IMobileServiceSyncTable (.NET client SDK) or MSSyncTable (iOS client). All the same CRUD operations (Create, Read, Update, Delete) work against sync table APIs, except now they read from or write to a local store. Before any sync table operations can be performed, the local store must be initialized.

What is a local store?

A local store is the data persistence layer on the client device. The Azure Mobile Apps client SDKs provide a default local store implementation. On Windows, Xamarin and Android, it is based on SQLite. On iOS, it is based on Core Data.

To use the SQLite-based implementation on Windows Phone or Microsoft Store, you need to install a SQLite extension. For more information, see Universal Windows Platform: Enable offline sync. Android and iOS ship with a version of SQLite in the device operating system itself, so it is not necessary to reference your own version of SQLite.

Developers can also implement their own local store. For instance, if you wish to store data in an encrypted format on the mobile client, you can define a local store that uses SQLCipher for encryption.

What is a sync context?

A sync context is associated with a mobile client object (such as IMobileServiceClient or MSClient) and tracks changes that are made with sync tables. The sync context maintains an operation queue, which keeps an ordered list of CUD operations (Create, Update, Delete) that is later sent to the server.

A local store is associated with the sync context using an initialize method such asIMobileServicesSyncContext.InitializeAsync(localstore) in the .NET client SDK.

How offline synchronization works

When using sync tables, your client code controls when local changes are synchronized with an Azure Mobile App backend. Nothing is sent to the backend until there is a call to push local changes. Similarly, the local store is populated with new data only when there is a call to pull data.

• Push: Push is an operation on the sync context and sends all CUD changes since the last push. Note that it is not possible to send only an individual table's changes, because otherwise operations could be sent out of order. Push executes a series of REST calls to your Azure Mobile App backend, which in turn modifies your server database.
• Pull: Pull is performed on a per-table basis and can be customized with a query to retrieve only a subset of the server data. The Azure Mobile client SDKs then insert the resulting data into the local store.
• Implicit Pushes: If a pull is executed against a table that has pending local updates, the pull first executes a push() on the sync context. This push helps minimize conflicts between changes that are already queued and new data from the server.
• Incremental Sync: the first parameter to the pull operation is a query name that is used only on the client. If you use a non-null query name, the Azure Mobile SDK performs an incremental sync. Each time a pull operation returns a set of results, the latest updatedAt timestamp from that result set is stored in the SDK local system tables. Subsequent pull operations retrieve only records after that timestamp.
  To use incremental sync, your server must return meaningful updatedAt values and must also support sorting by this field. However, since the SDK adds its own sort on the updatedAt field, you cannot use a pull query that has its own orderBy clause.
  The query name can be any string you choose, but it must be unique for each logical query in your app. Otherwise, different pull operations could overwrite the same incremental sync timestamp and your queries can return incorrect results.
  
  

   await todoTable.PullAsync("todoItems" + userid,
        syncTable.Where(u => u.UserId == userid));

If you want to opt out of incremental sync, pass null as the query ID. In this case, all records are retrieved on every call to PullAsync, which is potentially inefficient.

Purging: You can clear the contents of the local store using IMobileServiceSyncTable.PurgeAsync. Purging may be necessary if you have stale data in the client database, or if you wish to discard all pending changes.A purge clears a table from the local store. If there are operations awaiting synchronization with the server database, the purge throws an exception unless the force purge parameter is set.

As an example of stale data on the client, suppose in the "todo list" example, Device1 only pulls items that are not completed. A todoitem "Buy milk" is marked completed on the server by another device. However, Device1 still has the "Buy milk" todoitem in local store because it is only pulling items that are not marked complete. A purge clears this stale item.

Source : https://docs.microsoft.com/en-us/azure/app-service-mobile/app-service-mobile-offline-data-sync