Mark Forrester

Forum Posts

• January 6, 2010 8:51 PM GMT

  in the topic Moving Data Migration out of the Silo in the forum OSS

  In my experience, a good Data Migration team will prove themselves to be a key component to the to all aspects of you application delivery.  Vendors are required to deploy stable and correct application & metadata builds.  System integrators are required to work with these vendors to ensure environment builds contain the correct application and metadata builds at the right time.

   

  Vendors should have a strong, repeatable application build process to ensure that each environment build contains the correct metadata build, application build, indexes, sequences, foreign keys, etc.  What if the environment build does not match the release note they produce?

   

  A good data migration team will define their own set of environment acceptance tests to validate each environment build.  They can use their intimate knowledge of the target database to validate sequences, indexes, foreign keys, table spaces, and metadata configuration for each environment build.  Missing indexes, foreign keys, etc. will often result in significant performance issues or fail transformation rules and can be quickly detected.  Data migration teams will often use parts of the application for their own test cases and can therefore also be used to validate the application build as well.

   

  Do not underestimate the capabilities of your data migration team.  A strong data migration team will likely be willing to help identify issues with the environment builds to aid the overall solution delivery, as well as their own.

• November 18, 2009 11:13 PM GMT

  in the topic Managing Data Quality in the forum OSS

  Normal 0 false false false EN-AU X-NONE X-NONE MicrosoftInternetExplorer4

  Ideally, data quality should be part of a business strategy with defined processes for managing their information sets.  However, data quality is often overlooked until a problem arises.  In my experience, these problems often bubble up to the surface during data transformation projects.  As a result, data quality often becomes an IT issue.
  

  Data transformation is responsible for moving and reporting data from A to B.  Generally speaking, most data can be transformed using a fairly simple set of common transformation rules.  Very specific and complex business rules are often required to target very specific data inconsistencies.  Data inconsistencies that cannot be resolved via business rules are reported to a data quality team to reconciling these errors.  Question is, how should these issues be managed?  What communication channels are required?  How are the issues prioritized?  Who is tracking progress and driving progress?  Data transformation understands their data requirements.  The data quality team knows how to fix the data, but does not know what data should be targeted.  The business understands the how the data impacts their business operations.
  

  My experience is that a separate steering committee needs to be adopted so they can provide governance across the three channels (transformation, data quality, and business).  The steering committee should be responsible for defining communication tools to track and report data quality issues (may be a web site, spreadsheet, or custom software), working with the business to prioritize issues, establishing timelines for the data quality team, and follow up with data transformation to validate the corrections.
  

• November 17, 2009 11:09 PM GMT

  in the topic The Art of Reconciliation in the forum OSS

  I have experienced this problem on a number of transformation projects. To re-enforce the point made by Monty, data transformation is almost never a 1:1 mapping between legacy objects and target objects. This complicates your ability to track the success rate of your transformation project. My recommendation is to identify the reporting metrics during the design and analysis phase of your project. These designs need to be reviewed by the development and test teams so that both teams have a similar view of what data is being reported. Defects are often raised according to the test teams report metrics, and being able to relate these back to the report metrics communicated to the business can be very valuable.