Skip to main content
Back to the blog
Published on
min read
4
About
Ismael Baddich
Data migrationDynamics 365Azure

Migrating an on-premise CRM to Dynamics 365 without losing your data

Lessons from a banking migration off CRM 2015 on-premise onto Dynamics 365: mapping, Azure Data Factory pipelines, reconciliation, and the traps that cost real money.

A CRM migration is judged on a single number: how much data did you lose? On a project moving a bank off Microsoft CRM 2015 on-premise onto Dynamics 365, that number was 0.28% — and every one of those records was identified, documented and signed off with the business before go-live.

Here’s the method that got us there.

Start with the mapping, not the tool

The most common mistake is opening Azure Data Factory on day one. A badly targeted pipeline will migrate wrong data very efficiently.

Before writing a single line, three questions need answers from the business:

  1. Which entities are actually in use? On a ten-year-old CRM, it’s normal to find that 30% of custom entities hold nothing alive. Don’t migrate your technical debt.
  2. What are the dependencies? Accounts before contacts, contacts before opportunities. An explicit dependency graph determines pipeline order.
  3. What is authoritative? If a field exists in the CRM and in a third-party system, decide now which one wins. Not during cutover weekend.

This phase produces a mapping document: source entity, target entity, field by field, transformation rule, business owner. It’s tedious. It’s also the difference between 0.28% and 8%.

Azure Data Factory for extract and load

Once the mapping is frozen, ADF does the heavy lifting. The structure that works:

  • Extract: one pipeline per source entity, copying into a SQL staging area. No transformation here — you want a faithful, timestamped, replayable copy of the source.
  • Transform: cleanup logic in SQL, inside staging. Deduplication, format normalisation, resolving lookups to the new GUIDs.
  • Load: write into Dataverse.

Splitting into three distinct stages isn’t cosmetic. When a transformation rule changes three days before go-live — and it will — you replay Transform and Load without re-pulling the source for six hours.

Keep staging online after go-live. For weeks afterwards, every “where did this record come from?” question is answered with one SQL query.

PowerShell for what ADF does badly

ADF is excellent at volume. It’s painful for fine-grained work: recreating a handful of records, fixing a lookup, replaying one specific batch.

For that, PowerShell with PsDataverse is far faster to write. The scripts that earned their keep:

  • Source/target record reconciliation with the deltas exported to CSV
  • Bulk owner reassignment after import
  • Targeted deletion of a failed batch, by batch id

One thing matters more than the rest: tag every migrated record with a batch id and its source key. A custom text field is enough. Without it, partial rollback is impossible and you replay everything.

Reconciliation is not optional

It’s the step compressed first when the plan slips, and that’s a mistake.

Three levels of control, cheapest to most expensive:

Level Control Catches
1 Record counts per entity, source vs target Missing batches
2 Checksums on numeric and date fields Broken transformations
3 Random sample validated by hand with the business Mapping misunderstandings

Level 1 is automatable and replays on every iteration. Level 3 rarely finds much — but what it finds is usually a misunderstanding of the business domain, the kind of error that shows up in no log anywhere.

The traps that cost money

GUIDs don’t survive. Dataverse generates new identifiers. Any logic referencing a hard-coded GUID — integration, report, Word template — breaks. Inventory them beforehand, not afterwards.

Dates have no timezone. CRM 2015 on-premise often stores server local time; Dataverse works in UTC. One missed conversion shifts an entire database by an hour or two. It’s silent, and nobody notices until the first monthly report.

Calculated fields recalculate. If your source holds a frozen historical value in a field that is calculated in the target, Dataverse will overwrite it. Migrate into a simple field, or accept losing the history.

Plugins fire on import. Unless explicitly disabled, every create triggers your plugins, workflows and Power Automate flows. On 400,000 records, that means 400,000 notification emails. Disable, import, re-enable — and test that the re-enable actually worked.

What to take away

A successful migration is boring. No heroics on cutover weekend, no improvised fixes at 3am. It’s boring because the hard work — the mapping, the transformation rules, the reconciliation — happened weeks earlier, in a spreadsheet nobody wanted to read.

That’s where the 0.28% is won.

Related articles