Thursday, July 9, 2026

Enhancing Model-Driven App Views with Custom Column Icons

Enhancing Model-Driven App Views with Custom Column Icons

A Hidden Feature That Can Make Your Views Much More User-Friendly

I came across a feature that I had never used before, even after working with Dynamics CRM and Power Platform for many years. While editing a column in a Dataverse view, I noticed two properties:

  • Web Resource
  • Function Name

At first, I thought Microsoft had introduced a new feature that allows custom rendering for individual view columns. After researching further, I learned that this capability has actually existed for quite some time, but it has remained somewhat hidden and underused.

Once I noticed it in the modern Power Apps maker experience, I decided to test it immediately. Within a few minutes, I was able to replace the standard Yes/No text in my Is Mandatory? column with custom icons.

The result was simple, but very effective. The view became much easier to scan visually.


What Is This Feature?

This feature allows a Dataverse view column to call a JavaScript function from a web resource. That JavaScript function can return an image web resource and tooltip text.

Dataverse then displays that image next to the column value in the view.

[
  "ImageWebResourceName",
  "Tooltip Text"
]

It is lightweight, simple, and useful for adding visual indicators to Model-driven App views.


A Simple Example

Normally, a view might display something like this:

Question Is Mandatory?
Check Pressure Yes
Enter PSI No
Verify Seal Yes

After configuring custom icons, the same view can display:

Question Is Mandatory?
Check Pressure ✅ Yes
Enter PSI ❌ No
Verify Seal ✅ Yes

The underlying data does not change. Only the visual representation changes.


How Does It Work?

The JavaScript function receives row data from Dataverse. Based on the value in that row, it returns the appropriate image web resource.

function showMandatoryIcon(rowData, userLCID) {

    var row = JSON.parse(rowData);

    var isMandatory =
        row.insp_ismandatory_Value ??
        row.insp_ismandatory;

    if (isMandatory === true || isMandatory === "true" ||
        isMandatory === 1 || isMandatory === "1") {

        return ["insp_yes", "Mandatory Question"];
    }

    return ["insp_no", "Optional Question"];
}

In this example:

  • insp_yes is the image web resource for Yes.
  • insp_no is the image web resource for No.
  • Mandatory Question and Optional Question are tooltip values.

Implementation Steps

Step 1: Create Image Web Resources

Create image web resources in your Dataverse solution.

insp_yes
insp_no

These can be SVG or PNG image web resources.

Step 2: Create a JavaScript Web Resource

Create a JavaScript web resource and add your function.

function showMandatoryIcon(rowData, userLCID) {
    try {
        var row = JSON.parse(rowData);

        var isMandatory =
            row.insp_ismandatory_Value ??
            row.insp_ismandatory;

        if (isMandatory === true || isMandatory === "true" ||
            isMandatory === 1 || isMandatory === "1") {

            return ["insp_yes", "Mandatory Question"];
        }

        return ["insp_no", "Optional Question"];
    }
    catch (e) {
        console.error("showMandatoryIcon failed:", e);
        return ["insp_no", "Icon error"];
    }
}

Step 3: Configure the View Column

Open the table view in the maker portal and select the column where you want the icon to appear.

In the column properties, configure:

Web Resource:
Your JavaScript web resource

Function Name:
showMandatoryIcon

Then save and publish your customizations.


Debugging Tip

One challenge is identifying the exact property name Dataverse passes into the JavaScript function.

To inspect the row data, temporarily add console logs:

function showMandatoryIcon(rowData, userLCID) {

    console.log("Raw rowData:", rowData);

    var row = JSON.parse(rowData);

    console.log("Parsed row object:", row);

    return ["insp_yes", "Test"];
}

Open browser developer tools using F12, go to the Console tab, and refresh the view. This will show the row object and available field names.


Practical Business Scenarios

This feature can be useful in many Model-driven App scenarios.

Inspection Status

  • 🟢 Completed
  • 🟡 In Progress
  • 🔴 Overdue

Pass / Fail

  • ✔ Pass
  • ✖ Fail

Priority

  • ⬇ Low
  • ➡ Medium
  • ⬆ High

Device Health

  • 🟢 Healthy
  • 🟠 Needs Service
  • 🔴 Out of Service

Approval Status

  • ⏳ Pending
  • ✔ Approved
  • ✖ Rejected

Question Types

  • ☑ Yes / No
  • 📝 Text
  • 📅 Date
  • 🔢 Number
  • 📋 Choice

Important Limitations

This is a very useful feature, but it is important to understand its limitations.

1. It Cannot Render HTML

You cannot return custom HTML, CSS, buttons, links, progress bars, or formatted text. The function only returns an image web resource and tooltip.

2. The Icon Is Not Clickable

The icon cannot open a dialog, execute JavaScript, navigate to another page, or trigger Power Automate. It is only a visual indicator.

3. It Does Not Work in Editable Grids

This feature is intended for read-only grids. If the view is using an editable grid, the custom icons will not display the same way.

4. No Cell Formatting

You cannot change the font color, background color, row color, alignment, padding, or row height.

5. No Interactive Controls

You cannot embed buttons, checkboxes, toggles, dropdowns, or input controls.

6. Images Must Be Web Resources

The image must already exist as a Dataverse image web resource. You cannot return external URLs, base64 images, or dynamically generated images.

7. Limited to One Image Per Column Cell

The function returns one image and one tooltip. It is not meant for displaying multiple icons inside a single cell.

8. Performance Matters

The JavaScript function runs once for every visible row in the view. Keep the logic lightweight and avoid expensive operations.

9. Not a Replacement for PCF

If you need rich UI features like badges, progress bars, charts, clickable buttons, custom layouts, or interactive components, a Power Apps Component Framework control is still the better option.


When Should You Use This Feature?

This feature is best used when you want to make a view easier to scan without building a full custom control.

Sometimes a small visual cue, like a green checkmark or red warning icon, communicates status faster than plain text.


My Thoughts

This is one of those small Dataverse features that can make a big difference in user experience.

It will not replace PCF controls, dashboards, or custom pages. But for lightweight visual indicators in Model-driven App views, it is extremely useful.


References

Wednesday, April 29, 2026

Stop Burning Power Automate Runs While Testing Loops

If you have ever built a serious Power Automate flow, not just a demo flow, but something actually used by the business, you have probably seen this problem.

You test your flow once, and suddenly it starts processing hundreds of records.

  • Your run history becomes messy
  • Debugging becomes painful
  • You waste time and flow runs
  • You may even trigger unnecessary emails or HTTP calls

I have learned this from real Power Platform projects. Now, when I am testing an Apply to each loop, I do not let it process the full dataset during development.


The Real Problem

In Power Automate, many actions return arrays, such as:

  • SharePoint Get items
  • Dataverse List rows
  • Select
  • Filter array

Most of the time, we plug the full output directly into an Apply to each loop.

outputs('Select')

That is fine for production, but it is not ideal while testing.


The Simple Fix

During testing, limit the loop input using the take() expression.

take(outputs('Select'), 5)

This tells Power Automate to process only the first 5 items from the array.

You are not changing the structure of the flow. You are simply controlling how much data the loop processes while you test.


Real Example from My Work

In one of my Power Automate flows for a Water outage notification process, the flow checks SharePoint list items, validates attachments, builds a webhook payload, sends an HTTP request, and handles failure emails.

That flow includes logic like:

  • Checking whether a .txt attachment exists
  • Validating outage counts
  • Sending data to an external webhook
  • Handling HTTP success and failure responses
  • Sending error emails only to the development team

Now imagine testing that logic against hundreds of records.

That is not controlled testing. That is noise.

By using:

take(outputs('Select'), 5)

I can test only a few records first, confirm the logic, and then remove the limit when I am ready for full testing or production.


Another Example: Case Management System

I have also worked on model-driven Power Apps and Dataverse-based projects where flows and backend logic interact with case records, person records, validation rules, and business processes.

In those kinds of systems, testing against too many Dataverse rows at once can make troubleshooting harder.

For Dataverse List rows, the same idea works:

take(body('List_rows')?['value'], 5)

This lets me test with a small number of records before running the logic across the full dataset.


Why This Is Useful

  • No premium connector required
  • No major flow redesign
  • No extra variable needed
  • Works with almost any array
  • Makes debugging much easier

My Personal Testing Rule

When I am building or debugging a flow, I usually follow this pattern:

  1. Build the flow normally
  2. Use take() to limit the loop during testing
  3. Validate the logic with a small number of records
  4. Remove the limit before production use

Important Reminder

Do not forget to remove or update the take() expression before moving to production, unless your actual business requirement is to process only a limited number of records.

This is mainly a development and testing technique.


Final Thought

Power Automate is not always the problem. Sometimes the way we test our flows makes development harder than it needs to be.

For me, using take() during loop testing is a small habit that saves time, keeps run history cleaner, and makes troubleshooting much easier.

If you are building serious Power Automate flows, do not test large loops blindly.

Control the loop first. Scale later.

Friday, March 13, 2026

Power Automate Optimization: Filter Rows vs Trigger Conditions - When and Why to Use Each

Filter Rows vs Trigger Conditions in Power Automate

Filter Rows vs Trigger Conditions in Power Automate

Why your flow might not even start

Recently in a Power Platform WhatsApp group, someone asked a very good question about created/modified trigger filtering in Power Automate.

“Isn't Power Automate being hit either way? You're just deciding whether to continue or not. So you aren't really reducing the number of Power Automate executions with that filter?”

This is actually a very common misunderstanding, even among experienced Power Platform developers.

My answer in the group was:

Trigger Conditions are evaluated server-side before the flow starts. The system decides whether the flow should execute or not.

Because this topic comes up frequently, I thought it would be useful to write a short blog explaining:

  • How Trigger Conditions actually work internally
  • How Filter Rows works
  • Why both help reduce unnecessary flow executions

The Misconception: “The flow runs anyway”

Many developers assume the following sequence happens:

Record Updated

Then

Power Automate flow is triggered

Then

A condition inside the flow is checked

Then

The flow stops if the condition is not met

If this were true, then yes, the flow would still count as an execution.

But this is not how Trigger Conditions work.

What Actually Happens Internally

When you configure a Trigger Condition, the evaluation happens before the flow instance is created.

Record Updated

Then

Power Automate trigger receives the event

Then

Trigger Condition is evaluated on the server side

Then

If the condition is True

Then

Flow instance is created

Then

Actions execute

If the condition evaluates to False:

Record Updated

Then

Trigger Condition is evaluated

Then

Condition is False

Then

Flow instance is not created

Meaning:

  • The flow never runs
  • No flow execution is created
  • No actions execute
  • No API calls are consumed
This behavior applies to most modern event-based triggers such as Dataverse and SharePoint.

Trigger Conditions vs Conditions Inside the Flow

Scenario 1 — Condition inside the Flow

Trigger: When a row is modified

Then

Condition inside the flow: State = Texas

Then

If Yes, send an email

Here is the problem.

Every update still starts the flow when you use a normal Condition action inside the flow. The check happens only after the flow has already begun.

Update Event Flow Runs
Customer name changed Yes
Phone number updated Yes
Notes updated Yes
State changed to Texas Yes

Even if the condition fails, the flow still ran.

  • Wasted flow runs
  • Extra API calls
  • Unnecessary system load

Scenario 2 — Using Trigger Conditions

Trigger: When a row is modified

Trigger Condition:
@equals(triggerOutputs()?['body/address1_stateorprovince'], 'Texas')

With Trigger Conditions, the check happens before the flow instance is created. If the condition is false, the flow does not start.

Update Event Trigger Condition Result Flow Runs
Customer name changed False No
Phone number updated False No
Notes updated False No
State changed to Texas True Yes

Now the flow starts only when the target business condition is met.

Where Trigger Conditions Are Configured

  1. Open your flow
  2. Click the Trigger
  3. Click Settings
  4. Add your expression under Trigger Conditions
@equals(triggerOutputs()?['body/address1_stateorprovince'], 'Texas')

Multiple trigger conditions can be added, and they act as AND conditions.

What About “Filter Rows”?

If you are using Dataverse triggers, you will also see Filter Rows.

address1_stateorprovince eq 'Texas'

Filter Rows uses OData query syntax.

Unlike Trigger Conditions, Filter Rows works even earlier in the process.

In Dataverse, Filter Rows is applied at the event subscription level, meaning only matching events are sent to Power Automate.

Example:

address1_stateorprovince eq 'Virginia'
Record Updated Flow Triggered
State = Florida No
State = Virginia Yes

This prevents non-matching events from even reaching the flow trigger.

Filter Rows vs Trigger Conditions

Feature Filter Rows Trigger Conditions
Evaluation level Dataverse event subscription Power Automate trigger engine
Syntax OData query Workflow expression
Capabilities Simple column filters Complex logic such as and, or, empty, and comparisons
Best use case Record filtering Business rule validation

Filter Rows runs first, then Trigger Conditions.

In practice, both must evaluate to true for the flow to start.

Using Both Together

The best design is often to use both features together.

Example:

Filter Rows:
address1_stateorprovince eq 'Virginia'

Trigger Condition:
@equals(triggerOutputs()?['body/accountcategorycode'], 1)

In this example:

  • Filter Rows first limits the event to records where the state is Virginia
  • Trigger Condition then checks whether the account category matches the business rule
  • The flow starts only if both conditions are satisfied

Best Enterprise Design Pattern

In high-volume Dataverse environments, the best trigger configuration usually looks like this:

Trigger

Then

Select Columns

Then

Filtering Columns

Then

Filter Rows

Then

Trigger Conditions
  • Select Columns reduces payload size
  • Filtering Columns ensures the flow triggers only when specific fields change
  • Filter Rows limits which records generate events
  • Trigger Conditions applies the final business rule before the flow starts

Real Enterprise Example

Imagine a customer table where you only want to notify a team when a record belongs to Virginia.

Poor design:

Trigger: When a row is modified

Then

Condition inside flow:
If State = Virginia

Then

Send notification

This runs every time the record is edited, even when the state is not Virginia.

Better design:

Filter Rows:
address1_stateorprovince eq 'Virginia'

And optionally:

Trigger Condition:
@not(empty(triggerOutputs()?['body/emailaddress1']))

Now the flow starts only when:

  • The updated record belongs to Virginia
  • The record has the business data needed for the next step

SharePoint Example

SharePoint does not offer Dataverse-style Filter Rows in the same trigger experience, but it does support Trigger Conditions.

Example:

@equals(triggerOutputs()?['body/State'], 'California')

This means the flow starts only when the SharePoint item has a State value of California.

Common Mistakes

Using Conditions inside the flow for basic filtering

Trigger

Then

Condition

Then

Exit

This wastes executions because the flow has already started.

Ignoring Trigger Conditions

Many developers simply do not realize this feature exists.

Putting complex logic in Filter Rows

Filter Rows supports OData syntax only. It is not the place for full workflow expressions.

Not using Filtering Columns

If the flow only cares about a few fields, configure the trigger to watch those fields instead of reacting to every update.

Performance Impact

Using Filter Rows and Trigger Conditions properly helps:

  • Reduce flow runs
  • Reduce API consumption
  • Improve performance
  • Prevent throttling
  • Improve maintainability

This becomes extremely important in enterprise environments where thousands of records may be updated daily.

Even when skipped triggers do not create real flow runs, very high event volumes can still affect throughput and responsiveness. Good trigger design still matters.

Final Takeaway

Coming back to the original WhatsApp question:

“Isn't Power Automate being hit either way?”

The answer is No.

Trigger Conditions are evaluated server-side before the flow instance is created.

If the condition evaluates to false, the flow never starts.

And if you are using Dataverse, Filter Rows can stop irrelevant events even earlier.

That is why Trigger Conditions and Filter Rows are critical optimization techniques when building scalable Power Automate solutions.


Tip: Before publishing, you may want to replace sample column names with the exact logical names from your own environment.

-- Warm Regards, Sudip Chakrovorty, M.S. Hand Ph.: +1-(614) 309-8282 (EDT)

Thursday, July 24, 2025

Understanding the 10 Linked Entity Limit in Dataverse FetchXML Queries (Dynamics 365 / CRM)

Understanding the 10 Linked Entity Limit in Dataverse FetchXML Queries (Dynamics 365 / CRM)

If you’ve worked with Microsoft Dataverse (previously part of Dynamics CRM and now the data platform for Dynamics 365 and Power Apps), you’re likely familiar with FetchXML—a powerful XML-based query language used to retrieve data from Dataverse tables. FetchXML is particularly useful for building reports, creating custom views, and performing complex data retrieval operations in Dynamics 365 and Power Apps. However, like any technology, it comes with certain constraints. One such limitation is that Dataverse enforces a maximum of 10 linked entities in a single FetchXML query. In this post, we’ll explore what this limitation means, why it exists, and how you can design your queries to work effectively within this constraint.

What is FetchXML?

Before diving into the limitation, let’s briefly recap what FetchXML is. FetchXML is a proprietary query language developed by Microsoft for Microsoft Dataverse. It allows developers and administrators to query data across multiple tables (similar to database tables) and their relationships. Unlike SQL, which is text-based, FetchXML uses an XML structure to define queries. This makes it easy to construct queries programmatically and integrate them into Dynamics 365 applications, Power Apps, and other Power Platform solutions.

For example, a simple FetchXML query might look like this:

<fetch version="1.0" output-format="xml-platform" mapping="logical" distinct="false">
  <entity name="account">
    <attribute name="name" />
    <attribute name="accountnumber" />
  </entity>
</fetch>

This query retrieves the name and accountnumber attributes from the account entity. However, FetchXML’s real power comes from its ability to link multiple entities together to retrieve related data.

The 10 Linked Entities Limitation

When building more complex queries, you might need to retrieve data from several related entities. For instance, you might want to fetch data from an account, its related contacts, the opportunities tied to those contacts, and so on. Each of these relationships is represented as a linked entity in FetchXML.

However, Dynamics CRM imposes a hard limit: a single FetchXML query can include no more than 10 linked entities. This means that you cannot link more than 10 entities together in one query, no matter how they are nested or related.

It’s important to note that this limitation is specific to FetchXML. Other query methods in Dynamics CRM, such as the QueryExpression class in the SDK, do not have this same restriction. However, since FetchXML is widely used—especially in scenarios like report building and user-defined views—understanding this limitation is crucial.

Why Does This Limitation Exist?

You might be wondering why Microsoft implemented this restriction. The primary reason is performance. Each linked entity in a FetchXML query adds complexity to the underlying database query. As the number of linked entities increases, so does the potential for slow query execution, which can impact the overall performance of the CRM system.

By capping the number of linked entities at 10, Dataverse ensures that queries remain efficient and don’t consume excessive resources. This is especially important in a multi-tenant cloud environment like Dynamics 365, where system performance must be balanced across many users and organizations.

Illustrating the Limitation with an Example

To better understand the limitation, let’s consider a practical scenario. Suppose you’re building a report that requires data from the following entities:

  • account (the primary entity)
  • contact (linked to account)
  • opportunity (linked to contact)
  • lead (linked to opportunity)
  • quote (linked to opportunity)
  • order (linked to quote)
  • invoice (linked to order)
  • product (linked to invoice)
  • pricelevel (linked to product)
  • territory (linked to account)
  • systemuser (linked to territory)

In this case, you’re trying to link 11 entities together in a single FetchXML query. However, since the limit is 10 linked entities, this query would fail. You’d receive an error indicating that the query exceeds the maximum number of linked entities allowed.

Working Within the Limitation

So, what can you do if your query requires more than 10 linked entities? Here are a few strategies to work within this constraint:

1. Break Down the Query

One approach is to break your large query into smaller, more manageable queries, each with fewer than 10 linked entities. You can then combine the results programmatically or in your reporting tool. For example, you might run one query to retrieve data from the first 10 entities and a second query to retrieve data from the remaining entities, then merge the results based on a common key.

2. Use Multiple Queries with Filters

Another option is to use multiple queries with filters based on the results of previous queries. For instance, you could first query the account and its related entities up to the 10th link, then use the results to filter a second query that retrieves data from the additional entities.

3. Leverage Other Query Methods

If FetchXML’s limitation is too restrictive for your scenario, consider using other query options available in Microsoft Dataverse. For example, the QueryExpression class in the Dataverse SDK for .NET allows developers to build complex queries programmatically and provides greater flexibility for data retrieval. However, this approach requires custom development and is typically used in plugins, custom integrations, or server-side code rather than in standard views or reports within Dynamics 365 or Power Apps.

4. Optimize Your Data Model

In some cases, requiring more than 10 linked entities may indicate that your Dataverse data model could be optimized. Review your table relationships and evaluate whether the number of joins required in your queries can be reduced. For example, restructuring relationships, using calculated or rollup columns, or introducing supporting tables can help simplify queries and improve performance.

Conclusion

The 10 linked entities limitation in Dataverse FetchXML queries is an important constraint to consider when designing complex data retrieval operations. Although it may appear restrictive, the limitation helps maintain platform performance, query efficiency, and overall system stability. By understanding why this constraint exists and designing solutions that work within it—such as simplifying queries, using alternative query approaches, or optimizing your data model—you can build scalable and efficient solutions on the Power Platform.

While FetchXML remains a powerful and widely used query language in Microsoft Dataverse, it is not the only option for retrieving data. Developers can also use tools such as the Dataverse SDK, Web API, or QueryExpression depending on the scenario. For many common use cases in Dynamics 365 and Power Apps, however, working within the 10 linked entities limit is both practical and effective.

Saturday, June 28, 2025

Bulk Updating Over 20,000 Records in Dataverse ExecuteMultipleRequest —Step-by-Step Implementation

Mastering Dataverse Updates with ExecuteMultipleRequest: A Modern C# Guide

Dataverse Updates with ExecuteMultipleRequest

Recently, in a WhatsApp group I’m part of, someone reached out with a challenge: "With pagination, I want to update 20,511 records using Power Automate, but I am not able to do it more than 5,000. Anyone has any idea how to implement?" It’s a classic problem when working with Microsoft Dataverse—Power Automate, while great for many tasks, hits a wall with its pagination limits. In this case, the 5,000-record cap was stopping them cold, even with pagination tweaks.

Power Automate’s low-code simplicity is fantastic for small-to-medium workflows, but for bulk updates like this, it’s not the best fit. The pagination limit forces you into processing records in tiny batches, slowing things down and complicating the flow. For 20,511 records, that’s a lot of hoops to jump through—and a lot of time wasted.

My advice? Ditch Power Automate for this job and use a C# console application with ExecuteMultipleRequest. This Dataverse SDK method lets you bundle up to 1,000 operations into a single request, slashing the number of API calls and turbocharging performance. It’s a better choice because:

  • No Pagination Hassles: Unlike Power Automate, ExecuteMultipleRequest doesn’t choke on large datasets—you can process all 20,511 records without breaking them into 5,000-record chunks.
  • Faster Execution: Batching operations reduces server round trips, making it way quicker for big updates.
  • More Control: A console app lets you customize error handling, logging, and logic—things Power Automate struggles to match.

In this post, I’ll show you how to build that console app step-by-step, complete with code snippets and tips. Let’s get those records updated efficiently!

What You’ll Need

Before we start coding, gather these essentials:

  • Dataverse Environment: Access to a Dataverse instance with a table (e.g., account) containing records to update.
  • Azure AD App Registration: A registered app with client ID, client secret, and tenant ID, plus Dataverse permissions.
  • .NET Development Setup: Visual Studio or VS Code with .NET Core 3.1 or later.
  • NuGet Packages:
    • Microsoft.CrmSdk.CoreAssemblies
    • Microsoft.CrmSdk.XrmTooling.CoreAssembly
    • Microsoft.IdentityModel.Clients.ActiveDirectory

Step-by-Step: Building the Console App

Step 1: Create the Console Project

Fire up your terminal and create a new C# console app, then add the necessary NuGet packages.

dotnet new console -n DataverseBatchUpdater
cd DataverseBatchUpdater
dotnet add package Microsoft.CrmSdk.CoreAssemblies --version 9.0.2.34
dotnet add package Microsoft.CrmSdk.XrmTooling.CoreAssembly --version 9.1.0.68
dotnet add package Microsoft.IdentityModel.Clients.ActiveDirectory --version 5.3.0

Step 2: Set Up Authentication

To connect to Dataverse, configure authentication using your Azure AD app credentials. Open Program.cs and add these constants, replacing placeholders with your values.

private static readonly string dataverseUrl = "https://yourorg.crm.dynamics.com";
private static readonly string clientId = "your-client-id";
private static readonly string clientSecret = "your-client-secret";
private static readonly string tenantId = "your-tenant-id";
private static readonly string authority = $"https://login.microsoftonline.com/{tenantId}";

We’ll use these to authenticate via the Dataverse SDK’s CrmServiceClient.

Step 3: Authenticate with Dataverse

Add a method to establish a connection to Dataverse using the client credentials flow.

private static async Task GetCrmServiceClient()
{
    var authContext = new AuthenticationContext(authority);
    var clientCred = new ClientCredential(clientId, clientSecret);
    var authResult = await authContext.AcquireTokenAsync(dataverseUrl, clientCred);

    var connectionString = $"AuthType=ClientSecret;Url={dataverseUrl};ClientId={clientId};ClientSecret={clientSecret};RequireNewInstance=true";
    var serviceClient = new CrmServiceClient(connectionString);

    if (serviceClient.IsReady)
    {
        return serviceClient;
    }
    else
    {
        throw new Exception($"Connection failed: {serviceClient.LastCrmError}");
    }
}

Step 4: Prepare Records for Update

To use ExecuteMultipleRequest, we need a list of records to update. For this example, we’ll fetch accounts and modify their name field. Here’s how to query records (we’ll assume you have a pre-populated list or fetch them separately).

// Example: Fetch records (replace with your data source)
var accounts = new List();
for (int i = 1; i <= 20511; i++)
{
    var account = new Entity("account") { Id = Guid.NewGuid() };
    account["name"] = $"Account_{i}";
    accounts.Add(account);
}

In practice, you’d retrieve these using RetrieveMultiple, but we’ll focus on the update logic here.

Step 5: Implement ExecuteMultipleRequest

Now, the star of the show: ExecuteMultipleRequest. We’ll process records in batches of 1,000 (the max allowed per request).

private static async Task UpdateRecordsAsync(IOrganizationService service, List accounts)
{
    int batchSize = 1000;
    int totalUpdated = 0;

    for (int i = 0; i < accounts.Count; i += batchSize)
    {
        var requestCollection = new ExecuteMultipleRequest
        {
            Settings = new ExecuteMultipleSettings
            {
                ContinueOnError = true, // Keep going if one update fails
                ReturnResponses = true  // Get feedback on each operation
            },
            Requests = new OrganizationRequestCollection()
        };

        int endIndex = Math.Min(i + batchSize, accounts.Count);
        for (int j = i; j < endIndex; j++)
        {
            var entity = accounts[j];
            entity["name"] = $"{entity.GetAttributeValue("name")}_Updated";
            var updateRequest = new UpdateRequest { Target = entity };
            requestCollection.Requests.Add(updateRequest);
        }

        var response = (ExecuteMultipleResponse)service.Execute(requestCollection);
        totalUpdated += requestCollection.Requests.Count;

        foreach (var resp in response.Responses)
        {
            if (resp.Fault != null)
            {
                Console.WriteLine($"Error at index {i + resp.RequestIndex}: {resp.Fault.Message}");
            }
        }

        Console.WriteLine($"Updated {requestCollection.Requests.Count} records. Total: {totalUpdated}");
        await Task.Delay(500); // Throttling prevention
    }

    Console.WriteLine($"Finished! Total records updated: {totalUpdated}");
}

Tip 1: Batch Size Matters

Stick to 1,000 records per batch—Dataverse’s upper limit for ExecuteMultipleRequest. Smaller batches work but slow you down unnecessarily.

Step 6: Tie It All Together

Update Main to connect and run the update.

static async Task Main(string[] args)
{
    try
    {
        var service = await GetCrmServiceClient();
        Console.WriteLine("Connected to Dataverse!");

        // Replace with your record-fetching logic
        var accounts = new List();
        for (int i = 1; i <= 20511; i++)
        {
            var account = new Entity("account") { Id = Guid.NewGuid() };
            account["name"] = $"Account_{i}";
            accounts.Add(account);
        }

        await UpdateRecordsAsync(service, accounts);
    }
    catch (Exception ex)
    {
        Console.WriteLine($"Oops! {ex.Message}");
    }
}

Step 7: Run and Verify

Build and run your app, then check Dataverse to confirm the updates.

dotnet run

Look for console output like “Updated 1000 records” and verify in Dataverse that account names now end with "_Updated".

The Full Code

Here’s the complete C# console app. Replace placeholders with your Dataverse and Azure AD details.

using Microsoft.Crm.Sdk.Messages;
using Microsoft.IdentityModel.Clients.ActiveDirectory;
using Microsoft.Xrm.Sdk;
using Microsoft.Xrm.Tooling.Connector;
using System;
using System.Collections.Generic;
using System.Threading.Tasks;

namespace DataverseBatchUpdater
{
    class Program
    {
        private static readonly string dataverseUrl = "https://yourorg.crm.dynamics.com";
        private static readonly string clientId = "your-client-id";
        private static readonly string clientSecret = "your-client-secret";
        private static readonly string tenantId = "your-tenant-id";
        private static readonly string authority = $"https://login.microsoftonline.com/{tenantId}";

        static async Task Main(string[] args)
        {
            try
            {
                var service = await GetCrmServiceClient();
                Console.WriteLine("Connected to Dataverse!");

                // Simulated record list (replace with your fetch logic)
                var accounts = new List();
                for (int i = 1; i <= 20511; i++)
                {
                    var account = new Entity("account") { Id = Guid.NewGuid() };
                    account["name"] = $"Account_{i}";
                    accounts.Add(account);
                }

                await UpdateRecordsAsync(service, accounts);
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Oops! {ex.Message}");
            }
        }

        private static async Task GetCrmServiceClient()
        {
            var authContext = new AuthenticationContext(authority);
            var clientCred = new ClientCredential(clientId, clientSecret);
            var authResult = await authContext.AcquireTokenAsync(dataverseUrl, clientCred);

            var connectionString = $"AuthType=ClientSecret;Url={dataverseUrl};ClientId={clientId};ClientSecret={clientSecret};RequireNewInstance=true";
            var serviceClient = new CrmServiceClient(connectionString);

            if (serviceClient.IsReady)
            {
                return serviceClient;
            }
            else
            {
                throw new Exception($"Connection failed: {serviceClient.LastCrmError}");
            }
        }

        private static async Task UpdateRecordsAsync(IOrganizationService service, List accounts)
        {
            int batchSize = 1000;
            int totalUpdated = 0;

            for (int i = 0; i < accounts.Count; i += batchSize)
            {
                var requestCollection = new ExecuteMultipleRequest
                {
                    Settings = new ExecuteMultipleSettings
                    {
                        ContinueOnError = true,
                        ReturnResponses = true
                    },
                    Requests = new OrganizationRequestCollection()
                };

                int endIndex = Math.Min(i + batchSize, accounts.Count);
                for (int j = i; j < endIndex; j++)
                {
                    var entity = accounts[j];
                    entitytrong["name"] = $"{entity.GetAttributeValue("name")}_Updated";
                    var updateRequest = new UpdateRequest { Target = entity };
                    requestCollection.Requests.Add(updateRequest);
                }

                var response = (ExecuteMultipleResponse)service.Execute(requestCollection);
                totalUpdated += requestCollection.Requests.Count;

                foreach (var resp in response.Responses)
                {
                    if (resp.Fault != null)
                    {
                        Console.WriteLine($"Error at index {i + resp.RequestIndex}: {resp.Fault.Message}");
                    }
                }

                Console.WriteLine($"Updated {requestCollection.Requests.Count} records. Total: {totalUpdated}");
                await Task.Delay(500); // Avoid throttling
            }

            Console.WriteLine($"Finished! Total records updated: {totalUpdated}");
        }
    }
}

Pro Tips for Success

Tip 1: Maximize Batch Efficiency

Use the full 1,000-record capacity of ExecuteMultipleRequest to minimize requests and speed up execution.

Tip 2: Dodge Throttling Limits

Add a small delay (e.g., await Task.Delay(500)) between batches to avoid hitting Dataverse’s throttling limits.

Tip 3: Robust Error Handling

Log errors from response.Responses to a file or table for post-run analysis, ensuring no issues slip through.

Wrapping Up

With ExecuteMultipleRequest, updating large datasets in Dataverse becomes a breeze. This C# console app showcases how to harness batch processing for efficiency and reliability. Try it out, tweak it to your needs, and watch your Dataverse updates soar!

Friday, June 27, 2025

How to Retrieve More Than 5000 Records Using FetchXML and Power Automate

Power Automate: Paging Through 5000+ Records with FetchXML

When Dataverse limits you to 5,000 records per page, Power Automate gives you the tools to break past that limit—without writing C# or JavaScript. In this guide, you’ll create a Flow that loops through large datasets using FetchXML paging logic.

When Should You Use Power Automate for This?

  • You want to export large record sets to Excel, SharePoint, SQL, or external APIs
  • You want to schedule this data sync (e.g., nightly)
  • You don’t want to maintain plugin code or JS logic
  • You need a low-code way to pull 10k+, 50k+ records from Dataverse
💡 Pro Tip: Power Automate handles up to 100k rows using paging in loops. Just be sure to avoid infinite loops or huge memory loads!

Step 1: Create Your Base FetchXML

<fetch mapping="logical" count="5000">
  <entity name="account">
    <attribute name="accountid" />
    <attribute name="name" />
    <filter>
      <condition attribute="statecode" operator="eq" value="0" />
    </filter>
  </entity>
</fetch>

Step 2: Build the Flow

  1. Create a Manual trigger or use a Scheduled trigger
  2. Initialize these variables:
    • pagingCookie (string, empty)
    • pageNumber (int, default 1)
    • moreRecords (boolean, true)
    • recordsArray (array, empty)
  3. Add a Do Until loop:
    • Condition: moreRecords is equal to false

Step 3: Build Dynamic FetchXML with Pagination

Use a Compose action inside the loop to generate this:

<fetch mapping="logical" count="5000" page="@{variables('pageNumber')}"@{if(variables('pagingCookie'), concat(' paging-cookie="', variables('pagingCookie'), '"'), '')}>
  <entity name="account">
    <attribute name="accountid" />
    <attribute name="name" />
    <filter>
      <condition attribute="statecode" operator="eq" value="0" />
    </filter>
  </entity>
</fetch>
Tip: You can use concat() + if() to conditionally add the paging-cookie only when it exists.

Step 4: Fetch and Merge Results

Inside the loop:

  • Add a “List rows using FetchXML” action (Dataverse connector)
  • Pass in the dynamic FetchXML from Compose
  • Use “Append to array variable” to merge value into recordsArray
append(outputs('List_rows_using_FetchXML')?['body']?['value'])

Step 5: Handle Paging State

  • Update moreRecords using an expression like:
@if(contains(outputs('List_rows_using_FetchXML')?['body'], '@odata.nextLink'), true, false)
  • Set pagingCookie from @odata.nextLink by parsing the query string (use substring, split, or parse approach)
  • Set pageNumber to add(variables('pageNumber'), 1)
⚠️ Warning: PagingCookie is encoded in @odata.nextLink. You may need to decode or extract from the query string using split() or uriComponentToString().

Step 6: Post-Loop Actions

Once the loop is done, your recordsArray contains ALL records. You can now:

  • Loop over them to create SharePoint rows
  • Insert into SQL or external APIs
  • Dump them into Excel via Create Table + Add a Row

Real-World Use Cases

  • Export nightly case updates to Excel
  • Push 100k record snapshots to Data Lake
  • Sync contacts to downstream email platform

Advanced Ideas

  • Convert flow into a Custom Connector wrapper with retry control
  • Trigger flow from a Model-Driven App button using HTTP Request trigger
  • Use Secure Inputs to store base FetchXML in Dataverse for reuse

How to Retrieve More Than 5000 Records Using FetchXML and Dynamics 365 Plugin

Server-Side FetchXML Paging Using a Dynamics 365 Plugin

If you're working server-side inside a Plugin or Custom Workflow Activity, you can bypass client-side paging limits and fully control how many records to pull using FetchXML and paging cookies.

When to Use a Plugin for This

  • Fetching large datasets for processing on the server
  • Validating or aggregating records during business logic
  • Keeping logic secure and non-reversible from the client
  • Integrating real-time, record-heavy automations

Step 1: Base FetchXML Structure

Start with this structure:

<fetch mapping='logical' count='5000'>
  <entity name='account'>
    <attribute name='accountid' />
    <attribute name='name' />
    <filter>
      <condition attribute='statecode' operator='eq' value='0' />
    </filter>
  </entity>
</fetch>
💡 Tip: Keep your base FetchXML free of paging logic—this will be dynamically injected.

Step 2: Plugin Implementation

This plugin fetches all active account records, 5000 at a time, using the PagingCookie and MoreRecords flags.

public class FetchAllAccountsPlugin : IPlugin
{
    public void Execute(IServiceProvider serviceProvider)
    {
        ITracingService tracer = (ITracingService)serviceProvider.GetService(typeof(ITracingService));
        IPluginExecutionContext context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
        IOrganizationServiceFactory factory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
        IOrganizationService service = factory.CreateOrganizationService(context.UserId);

        string pagingCookie = null;
        int pageNumber = 1;
        int fetchCount = 5000;
        bool moreRecords = true;
        List<Entity> allAccounts = new List<Entity>();

        while (moreRecords)
        {
            string fetchXml = CreateFetchXml(pagingCookie, pageNumber, fetchCount);
            EntityCollection results = service.RetrieveMultiple(new FetchExpression(fetchXml));

            allAccounts.AddRange(results.Entities);
            tracer.Trace($"Fetched page {pageNumber}: {results.Entities.Count} records");

            moreRecords = results.MoreRecords;
            if (moreRecords)
            {
                pagingCookie = results.PagingCookie;
                pageNumber++;
            }
        }

        tracer.Trace($"✅ Total records fetched: {allAccounts.Count}");
    }

    private string CreateFetchXml(string pagingCookie, int pageNumber, int fetchCount)
    {
        return $@"
<fetch mapping='logical' page='{pageNumber}' count='{fetchCount}'{(pagingCookie != null ? $" paging-cookie='{{System.Security.SecurityElement.Escape(pagingCookie)}}'" : "")}>
  <entity name='account'>
    <attribute name='accountid' />
    <attribute name='name' />
    <filter>
      <condition attribute='statecode' operator='eq' value='0' />
    </filter>
  </entity>
</fetch>";
    }
}
⚠️ Watch out: Don’t forget to escape the pagingCookie! Use System.Security.SecurityElement.Escape() to avoid breaking XML syntax.

Bonus Tips

  • Use an async plugin or Custom API for long-running tasks
  • Consider streaming results or batching when working with >50,000 records
  • For external systems, push the results into Azure or a Data Export Service

Registration Ideas

This plugin can be triggered via:

  • A Custom API (for on-demand calls)
  • A Workflow or Power Automate Flow
  • Or bound to Entity Events like Create/Update (with filters)

Enhancing Model-Driven App Views with Custom Column Icons

Enhancing Model-Driven App Views with Custom Column Icons A Hidden Feature That Can Make Your Views Much More User-Friendly I came acro...