White Paper Migrating Data into Remedyforce Getting Started with Remedyforce Series Dale Jones 24 March 2015 Data Migration Getting Started with Remedyforce Series Welcome to the “Getting Started with BMC Remedyforce” Series Today’s IT departments must drive business growth and innovation, while coping with less resources and increasing complexity. To do this, they require an IT Service Management solution that provides best practices while minimizing costs. BMC Remedyforce is built on Salesforce—the world's most widely used cloud platform—to deliver complete IT service management functionality with the secure social, mobile, and collaborative capabilities users expect. With the “Getting Started with Remedyforce” white paper series, our aim is to help you leverage BMC Remedyforce to improve the effectiveness and efficiency of your ITSM operations. Each paper addresses a specific area of interest and provides you with conceptual, functional and technical best practices to make configuration decisions and take action to gain value from your BMC Remedyforce investment. Remedyforce Data Migration When making a major change to an ITSM system, such as moving from one tool to another or adding new organizations to an existing tool, decisions have to be made about what to do with the information from another system. Should data be brought over into the new tool? Is the complexity of doing so too high to justify the investment in personnel and time? What data should be brought over, and what is best served by placing the data in a backup for as-needed access? The purpose of this white paper is to address data migration considerations, provide instructions on Apex Data Loader setup and provide some useful data cleanup formulas to make the migration of data easier. Data Migration Considerations Most often, organizations are faced with a decision about whether to migrate data when they are either implementing a new system, or onboarding an organization into an existing system. Data Migration: The movement of data from an existing system (legacy or prior) to a new system. The first thing to consider when faced with a migration decision is – WHY? What is the benefit of migrating the data? Some reasons for data migration could include: Historical Reporting – There is a need to access historical information on a frequent basis. Legal (or Regulatory Requirements) – Certain rules/regulation require retention of data in a readily accessible system for a period of time. Contractual Requirements – There may be customer or organizational contracts in place to have the data accessible for a period of time. Starting Point for a New System – Migration of open requests or last 3 months of data or system foundational data (eg. menus and organizational information). There are many reasons to migrate data to the new system, and just as many alternatives to migrating all the data. Archival Solution Option One way to save cost and time is to move the historical data to an archival solution. An archival solution does not mean data is not accessible and reportable. Most systems have their data stored in a database. A database copy can be moved to another database, which now can be accessed and reported on by hundreds of database reporting tools. Existing reports would just need to be tweaked to point to a new database location. With the Archival Solution approach, you can PAGE 2 OF 9 Copyright BMC Software, Inc. 2015 Data Migration Getting Started with Remedyforce Series still meet the requirements for historical reporting, legal and contractual requirements, while reducing the complexity for implementation and ongoing maintenance of the new solution. Archival does not mean data is not accessible. Archival simply means the standard reporting solution will need to pull data from one more source. Most reporting solutions can pull data from multiple sources. Partial Data Migration Option Another alternative to migrating all data is to only migrate part of the data. This will reduce the data migration effort and cost. This is a great option to get foundational data like groups and categorizations and also to migrate all open requests. Some companies also choose to migrate some closed records by choosing a key reporting date, like the start of a year or the start of a quarter or start of a month. (eg. Migrate all data created since January 1 2015) Implications of Data Migration Having a clean system at the start of go-live keeps the new system uncluttered and enables it to have the out-of-the-box (OOTB) best practices easily adopted without customization to accommodate the old data. Migrating all data from an existing system to the new system can double the amount of time to implement the new system and prepare it for production use, which will increase the budget for the project. Choosing to migrate all data often adds complexity because you need to add fields to the new system to store data in a different format (from the legacy system). This can change the screen layouts and user experience, meaning that help files and other OOTB tools will be out of sync with the customized solution. Whether you choose to do a data migration, a partial data migration, or archive your existing data, you should be informed about the implications and risks of each approach. Data Migration Best Practices There are many considerations when migrating data that can make a project more successful. The biggest impact on a data migration project is the volume of data. The more data to be moved the longer data will take to export, clean and migrate. The older pieces of data usually have more mapping issues to account for, as processes have usually changed over time. The older the data is, the less likely the data is clean. If key people with historical data knowledge are not around to properly map this data to a new system, there is a greater risk of more data issues to resolve. Another key question is whether the data being migrated from the “Single Source of Truth.” If the data is more accurate in another location, you should consider if it makes sense to move the data from a less authoritative location. This usually comes into play when migrating CMDB/Asset data, HR data or Active Directory data. It makes sense to go right to the source of the accurate information. Geographic and security discussions need to be held with key stakeholders to address questions like: Can the data ever leave the current country? Some solutions have backup servers or mirrored servers in other countries so it is important to know the rules/regulations regarding data storage and movement. Does data need to be encrypted if moved over an external network? Some industries (Banking, Governments, Pharmaceutical…) have security standards that have to be followed when transferring data. Another key step is to identify the data archival plan. This plan needs to account for the historical data in the current system and the future archival plan for historical data on the new system. All systems run more efficiently when the volume of data is reduced. The more data that can be archived and not moved to the new system, the smoother the new system will run in production. There are many things that can make the data migration project more successful, but the biggest success factor is planning early. Data migration takes time and cannot be rushed. Get the key people involved that have historical data knowledge and people that can make decisions for the future state of the data. Some ways to alleviate data mapping conflicts are to create a matching legacy field (like legacy category), or concatenate data into a larger text field (like ‘Description’). If an archival solution for the current system and the future system can be put in place, the project will save data migration time and budget. Make sure to have a unique identifier that is unique on the old and new system. This is PAGE 3 OF 9 Copyright BMC Software, Inc. 2015 Data Migration Getting Started with Remedyforce Series normally a new custom field that will hold the old systems unique record number. This will save a lot of time during data validation and speed up the cleanup of any data discrepancies. This unique identifier will also allow Administrators to do multiple delta data migration before the cutover to production. Data Migration Methodology Stage Notes 1. High Level Analysis, Design and Overview 2. Extraction Plan - 3. Data Cleanup - 4. Data Mapping - - - 5. Testing Plan - 6. Recovery Plan 7. Initial Data Load - 8. Delta Migrations - 9. PAGE 4 OF 9 Go live - Owner of Data Volume of Data Type of Data Sources of data Are Attachments being migrated Geographic Concerns Security Concerns Archival Solution How data is pulled from the existing system. What is pulled and what is not pulled from the existing system. How data is cleaned to match the new system requirements. If there are a lot of data issues with the current system. These issues need to be addressed before or during the migration. Nothing worse than migrating bad data. The old saying “Garbage In Garbage Out” New required fields but the old system does not always have a value in this field Data validation rules in new system that were not enforced in the legacy system Data size issues (Size limits on new system) (Plan to deal with large data records) Map the existing Application to the New Application o i.e. Existing ITSM System will be moved to Remedyforce Map the existing forms to the new forms o i.e. Help Desk data will go to Incident object in Remedyforce o i.e. People data will go to User Object in Remedyforce Map the existing fields to a new field or multiple fields on the new System o Incident # - External Ticket o Status – Status o Field A, B, C concatenated into Description o … How to validate the data was moved successfully Normally done with record counts and reports with counts How is data recovered or fixed if a discrepancy is found Leverage a Unique Identifier field for cleanup Leverage Data Loader Upsert or Update Load all needed data into the new system Make sure a unique identifier exists in both system (IE Ticket Number or Employee ID) This allows data to be validated in the new system months before go live This allows for data that has changed since initial Data Load to be loaded multiple times before the go live This allows for discrepancies or process changes to be address This allows for practice of the Go Live Delta migration multiple times This also limits the amount of data that is migrated during Go Live, which limits the outage and Go Live validation Old System switched to Read Only Final migration of the changed data since the last Delta Migration Copyright BMC Software, Inc. 2015 Data Migration Getting Started with Remedyforce Series Apex Data Loader There are many tools that can be leveraged for moving data in and out of Remedyforce. Data Loader is just one tool, but it has one advantage: Data Loader has a download link built right into the Salesforce platform. Instructions for using Data Loader 1. 2. PAGE 5 OF 9 Open Salesforce/Remedyforce Navigate to Setup, Administration, Data Management, Data Loader Copyright BMC Software, Inc. 2015 Data Migration Getting Started with Remedyforce Series 3. Press the “Download the Data Loader” link 4. Run Install exe 5. Get Security Token (Personal Setup, Reset My Security Token) 6. You will get an Email similar to the following: Dear Sample User, When accessing salesforce.com from outside of your company’s trusted networks, you must add a security token to your password to log in to a desktop client, such as Connect for Outlook, Connect Offline, Connect for Office, Connect for Lotus Notes, or the Data Loader. New security tokens are automatically sent to you when your salesforce.com password is changed or when you request to reset your security token. Your new security token is below. Note that security tokens are case sensitive. User Name: [email protected] Security Token: GwCiI0DdWHOMRai68lZVaPABC Please add your security token to your password. Note that you do not enter a security token in place of your password when logging into salesforce.com via a browser. ----For Outlook Edition and Connect for Outlook: 1. Go to Outlook > Tools > Salesforce Options 2. Enter the password and security token in the password window. For example, if your password is "myPassword" and your security token is "XXXXXXX", you will enter "myPasswordXXXXXXX" into the password window. 3. Click verify. This tests that you have correctly entered your password and token. 4. If verification succeeds, Click 'OK' PAGE 6 OF 9 Copyright BMC Software, Inc. 2015 Data Migration Getting Started with Remedyforce Series 7. 8. Launch Data Loader Login User Name: [email protected] Password/Security Token: myPasswordGwCiI0DdWHOMRai68lZVaPABC 9. Data Loader Settings 10. Data Loaders Functions Export – Export data Update – Update existing records Insert – Insert new record Upsert – Update if a matching record is found, if a matching record is not found then an insert new record occurs. Delete – Delete records Data Loader Useful Tips Apex Data Loader is a very powerful tool that can be very productive in the right hands and very destructive in the wrong hands. Always perform backups (Export) of data before any changes to data occurs. In Settings, the Batch size sometimes needs to be reduced if the data imports causes triggers to run on additional forms. This needs to be done when importing Remedyforce Changes or Remedyforce Tasks. Also in settings the Login server setting needs to be set to https://test.salesforce.com for Sandbox servers and https://login.salesforce.com for Production servers. The Time Zone in settings, needs to match the data’s time zone for the data that is about to be imported. PAGE 7 OF 9 Copyright BMC Software, Inc. 2015 Data Migration Getting Started with Remedyforce Series Data Cleanup Formulas When you’re preparing data for import into BMC Remedyforce and the Salesforce platform, there is some cleanup that has to be done to ensure an easy load. This includes ensuring the data formats are consistent, that lookup fields are defined and that data is correctly formatted for load into the destination tables. Date/Time Formatting Importing Dates into Salesforce is tricky, as Salesforce is expecting the dates to be in a particular format. The best way to see how Salesforce expects the date to be formatted is to export some existing data and mimic the date format when importing new data into Salesforce. Some date conversion example formulas: UNIX Time example = 1420651550 Start of UNIX Time = 1/1/70 12:00 AM Unix time Formula = “=1/1/70 12:00 AM +(1420651550/86400)” Example of UNIX Time = 1420651550 would convert to “1/7/2015 5:25:50 PM EST” Salesforce Date Format = “=TEXT(C9,"yyyy-mm-ddThh:mm:ss.000Z")” Example of Date/Time = “1/7/2015 5:25:50 PM EST” would convert to 2015-01-07T17:25:50.000Z Lookup Fields Salesforce uses a lot of lookup fields. The GUID value for the lookup field is needed during the import. The way to get the GUID value is to export the existing values with their GUID. In the import file a VLookup column will need to be added with a formula to perform the VLookup. Example to get the GUID of a User in the excel file usersALLUAT.csv. "=VLOOKUP(E2,usersAllUAT.csv!$A:$B,2,FALSE)" Concatenating Data Some salesforce fields need the Last Name and First Name together or the need to concatenate multiple fields into one Description or Notes field. Example of Concatenation: =CONCATENATE(A9," ",B9) In Summary There are many considerations when migrating data from an existing ITMS system into Remedyforce. Early, detailed planning for data migration is key to your success. BMC Remedyforce Professional Services specializes in complex data migrations. You can learn more by visiting BMC Remedyforce Consulting Services at: http://www.bmc.com/it-services/remedyforce-services.html Hopefully this document has helped to answer some of your most pressing questions about data migration planning and execution, but you can always get more information by visiting us at the BMC Remedyforce User Community at: bmc.com/communities. PAGE 8 OF 9 Copyright BMC Software, Inc. 2015 BMC delivers software solutions that help IT transform digital enterprises for the ultimate competitive business advantage. W e have worked with thousands of leading companies to create and deliver powerful IT management services. From mainframe to cloud to mobile, we pair high-speed digital innovation with robust IT industrialization—allowing our customers to provide amazing user experiences with optimized IT performance, cost, compliance, and productivity. We believe that technology is the heart of every business, and that IT drives business to the digital age. BMC – Bring IT to Life.
© Copyright 2017