Skip to end of metadata
Go to start of metadata

You are viewing an old version of this content. View the current version.

Compare with Current View Version History

« Previous Version 15 Next »

This documentation describe a solution to migration from TGNG Server / Datacenter to TGNG Cloud.

In order to migrate from TGNG Server / Datacenter, we build a script to help migrate issues data.

We’re excited to introduce a brand new in-app feature for migrating your grid data to the Atlassian Cloud—no scripts or advanced technical skills required!

For more details, please visit link.

Contents

TGNG Server/Datacenter → TGNG Cloud migration

Groovy Script

 Expand to see the script....
import com.atlassian.jira.issue.customfields.CustomFieldType
import com.atlassian.jira.bc.issue.search.SearchService;
import com.atlassian.jira.component.ComponentAccessor;
import com.atlassian.jira.jql.parser.JqlQueryParser;
import com.atlassian.jira.issue.CustomFieldManager;
import com.atlassian.jira.issue.fields.CustomField;
import com.atlassian.jira.user.util.UserManager;
import com.atlassian.jira.web.bean.PagerFilter;
import groovy.json.StreamingJsonBuilder;
import com.atlassian.jira.issue.Issue;
import groovy.json.JsonSlurper;
import groovy.json.*;


////////////////////////////////////////////
// ***** CHANGE VARIABLES HERE *****
////////////////////////////////////////////

// Authorization 
String authorizationStringServer = "Basic ==="
String authorizationStringCloud = "Basic ==="
String authorizationStringCloudAccountID = """Basic ==="""

//Jira BASE URL
String jiraBaseURLServer = "https://==="
String cloudInstanceName = "==="
String jqlQuery = "==="

////////////////////////////////////////////
// ***** CHANGE VARIABLES HERE *****
////////////////////////////////////////////

String baseURL =  "https://databuckets.net/rest/tgc/api/v1/grids"

CustomFieldManager customFieldManager = ComponentAccessor.getOSGiComponentInstanceOfType(CustomFieldManager.class)
def tgngFieldType = customFieldManager.getCustomFieldType("tge.cloud:idalko-table-grid-custom-field-key")
def tgngFieldList = customFieldManager.getCustomFieldObjects().findAll {it.getCustomFieldType() == tgngFieldType}

// Cloud URL
def jsonSlurperAccount = new JsonSlurper()
StringBuilder responses = new StringBuilder()

// Get all Cloud Grids
URL getGridsURL;
getGridsURL = new URL(baseURL);
URLConnection getGridsConnection = getGridsURL.openConnection();
getGridsConnection.setRequestMethod("GET")
getGridsConnection.doOutput = true
getGridsConnection.setRequestProperty("Authorization", authorizationStringCloud)
getGridsConnection.connect();
def getGridsResponseCode = getGridsConnection.getResponseCode()
def gridsList
if(getGridsResponseCode != 200){
    responses.append("Error while getting the list of Grids in Cloud")
}else{
    gridsList = jsonSlurperAccount.parseText(getGridsConnection.getInputStream().getText()); 
}
Map<Long,String> gridCorrespondance = [:]


for(grid in tgngFieldList){
    if(gridsList.find { it.name == grid.toString() } != null) {
        gridCorrespondance.putAll([(grid): (gridsList.find { it.name == grid.toString() }.id)])
    }
}

def searchService = ComponentAccessor.getComponent(SearchService)
def loggedInUser = ComponentAccessor.getJiraAuthenticationContext().getLoggedInUser()
def queryParser = ComponentAccessor.getComponent(JqlQueryParser)
def sb = new StringBuffer()

//your query goes here
def query = queryParser.parseQuery(jqlQuery)

//gets results of query
def search = searchService.search(loggedInUser, query, PagerFilter.getUnlimitedFilter())

//iterate over each returned issue
Collection<Issue> issues = search.results.key

for (String issue : issues){
    
    for (tgngCustomField : gridCorrespondance) {

        String issueKey = issue
        // Grid IDs 
        String gridCustomFieldIdServer = tgngCustomField.getKey().getIdAsLong()
        String gridIdCloud = tgngCustomField.getValue()
        
        def gridDataDCURL = jiraBaseURLServer + "/rest/idalko-grid/1.0/api/grid/" + gridCustomFieldIdServer + "/issue/" + issueKey;
        def gridDataCloudURL =  "https://databuckets.net/rest/tgc/api/v1/grids/" + gridIdCloud + "/issue/" + issueKey;

        //Getting the Grid information from the Data Center Jira instance
        URL url;
        url = new URL(gridDataDCURL);

        URLConnection connection = url.openConnection();
        connection.setRequestMethod("GET")
        connection.doOutput = true
        connection.setRequestProperty("Authorization", authorizationStringServer)
        connection.connect();

        String serverData = connection.getInputStream().getText()
        def jsonSlurper = new JsonSlurper()
        def object = jsonSlurper.parseText(serverData)

        Long gridRowsinCloud = -1; //create variable with invalid value just to signal exception and skip condition if failed

        try {
            // Geting Grid info from Cloud to see if it was already uploaded (avoid duplicates)

            URL url3;
            url3 = new URL(gridDataCloudURL);
            URLConnection connection3 = url3.openConnection();
            connection3.setRequestMethod("GET")
            connection3.doOutput = true
            connection3.setRequestProperty("Authorization", authorizationStringCloud)
            connection3.connect();

            responseCode = connection3.getResponseCode()

            if(responseCode != 200){
                responseMessage = connection3.getErrorStream().text;
            }else{
                String cloudData = connection3.getInputStream().getText();
                def cloudjson = jsonSlurperAccount.parseText(cloudData)
                //Long gridRowsinServer = object.rows.size();
                gridRowsinCloud = cloudjson.rows.size(); //if it works, change value of variable
            }


        } catch(java.io.IOException ex) { responses.append("!! exception while getting Grid data !!"); }
    
        // check if data was already uploaded    
        if(gridRowsinCloud <= 0){

            // Formatting the output so that it fits the POST request body formatting requirements
            object.remove("customFieldId")         //Removing Grid ID since does not need to be passed in the request body
            object.rows.each { e ->         //Going through all of the rows from the response


                e.remove("order")           // Removing order index
                e.remove("rowId")           // Removing Row ID since it will be automatically assigned
                e.columns.each{ key, value ->   // Unpacking the "columns" section into the previous level of JSON structure
                    
                    if(value != null || !value.toString().equals("{}") || value != ""){

                        if(value instanceof String){
                            e.put(key, value)
                        }else if(value instanceof org.apache.groovy.json.internal.LazyMap){

                            if(value.keySet().contains("username")){
                            
                                // Get user email to match with user in Cloud
                                def userManager = ComponentAccessor.getUserManager()
                                def user = userManager.getUserByName(value.username)

                                if(user != null){
                                    String email = user.getEmailAddress()

                                    // Getting new user info from Cloud based on email
                                    URL urlAccountID
                                    urlAccountID = new URL("https://${instanceName}.atlassian.net/rest/api/3/user/search?query=" + email.replace(" ","%20"));
                                    URLConnection connection4 = urlAccountID.openConnection();
                                    connection4.setRequestMethod("GET")
                                    connection4.setRequestProperty("Authorization", authorizationStringCloudAccountID)
                                    connection4.doOutput = true
                                    connection4.connect();

                                    String cloudData1 = connection4.getInputStream().getText()
                                    def jsonSlurperAccount1 = new JsonSlurper()
                                    def objectAccount1 = jsonSlurperAccount1.parseText(cloudData1)
                                    
                                    if(objectAccount1[0] != null){
                                        e.put(key, jsonSlurperAccount1.parseText('{"accountId":"' + objectAccount1[0].accountId+'"}'))
                                    }
                                }
                            } 
                            if(value.keySet().contains("label")){
                                e.put(key, value.label)
                            }
                        }else if(value.toString().length() != 2){
                            e.put(key, value.value)
                        }
                    }
                }
                e.remove("columns")             // Removing the packed "columns" section as it is redundant and breaks the formatting
                e.remove("Item Number")         // Remove SEQUENCE type column from the data load 
            }


            try {

                // Uploading the grid data to the Cloud
                URL url2;
                url2 = new URL(gridDataCloudURL);
                URLConnection connection2 = url2.openConnection();
                connection2.setRequestMethod("POST")
                connection2.doOutput = true
                connection2.setRequestProperty("Authorization", authorizationStringCloud)
                connection2.setRequestProperty("Content-Type", "application/json;charset=UTF-8")
                connection2.outputStream.withWriter("UTF-8") { new StreamingJsonBuilder(it, object) }
                connection2.connect();

                responseCode = connection2.getResponseCode()

                if(responseCode != 201){
                    responseMessage = connection2.getErrorStream().text + JsonOutput.toJson(object);
                }else{
                    responseMessage = connection2.getResponseMessage();
                }

                responses.append(   " ||  IssueKey:  "   + issueKey + 
                                    " Grid Field Name: " + tgngCustomField.getKey() +
                                    " ResponseCode: "    + responseCode + 
                                    " ResponseMessage: " + responseMessage)

            } catch(java.io.IOException ex) {

                responses.append("!! exception !!");
            
            }

            responses.append("\n")

        }
    }
}

return responses

Prerequisites

  • A plugin/tool for running Groovy code is present on the instance from which the script is being run

  • TGNG apps installed on both Server and Cloud instances

  • The project from the source instance was already migrated to the target instance

  • All Columns in the Grid Config are made editable before running the script (can be changed after)

  • Please note that the script was written to be used as a sample and requires you to change the variables' values

  • None of the Target Grids have default rows (they can be re-added after the data is migrated)

Disclaimers

  • The script was developed to be run from the source Server instance.

  • The script is not an official solution for migration but it’s a workaround solution and should be used with care and precaution.

Guide

  1. Make sure that all of the prerequisites above are met

  2. Verify whether the grid fields on your Server instance have a multi-context setup

    1. If not, configure identical Grids on the Cloud instance (+ Scopes)

    2. If so, create an identical Grid configuration for each distinct context

  3. Obtain valid authentication strings from both instances

  4. Replace the placeholder authentication values as well as instance URLs with valid ones

  5. Replace the gridCustomFieldIdServer and gridIdCloud for the values corresponding to the first grid or Grid context that is being migrated

  6. Prepare the script to verification by adding a logic gate in the "for" loop or by hard-coding a single issue KEY into the issueKey variable

  7. Change projectId to a project that is relevant to the current grid field

  8. Verify whether object.rows.each { } loop does not contain any mentions of columns that are not present in the Grid configuration and add e.remove("COLUMN_NAME") for all sequence columns

  9. Run the script and verify data, for each Grid or Grid context:

    1. Change the gridCustomFieldIdServer, gridIdCloud, projected or any custom issue-pulling logic to relevant values

    2. Re-do step 11

    3. Run the script

    4. Verify whether the data was migrated correctly

TGE Server/Datacenter → TGNG Cloud migration

Groovy Script

 Expand to see the script....

 Click here to expand...
import groovy.json.JsonSlurper;
import groovy.json.StreamingJsonBuilder;
import com.atlassian.jira.issue.fields.CustomField;
import com.atlassian.jira.issue.CustomFieldManager;
import com.atlassian.jira.component.ComponentAccessor;
import com.atlassian.jira.issue.Issue;
import com.atlassian.jira.bc.issue.search.SearchService;
import com.atlassian.jira.jql.parser.JqlQueryParser;
import com.atlassian.jira.web.bean.PagerFilter;
import groovy.json.*;
import com.atlassian.crowd.embedded.api.User
import com.atlassian.jira.security.JiraAuthenticationContext
import com.atlassian.plugin.PluginAccessor
import com.atlassian.jira.user.ApplicationUser
 
////////////////////////////////////////////
// ***** CHANGE VARIABLES HERE *****
////////////////////////////////////////////
 
String authorizationStringCloud = "Basic ==="
 
String jqlQuery = '===='
 
////////////////////////////////////////////
// ***** CHANGE VARIABLES HERE *****
////////////////////////////////////////////
 
 
// find TGE custom fields
PluginAccessor pluginAccessor = ComponentAccessor.getPluginAccessor();
CustomFieldManager customFieldManager = ComponentAccessor.getComponent(CustomFieldManager);
Class tgeConfigManagerClass = pluginAccessor.getClassLoader().findClass("com.idalko.jira.plugins.igrid.api.config.grid.TGEGridConfigManager");
def tgeConfigManager = ComponentAccessor.getOSGiComponentInstanceOfType(tgeConfigManagerClass);
List<Long> tgeCustomFieldIds = tgeConfigManager.getGridCustomFieldIds();
List<Map<Long,String>> tgeCustomFieldNames = tgeCustomFieldIds.collect { [(it) : customFieldManager.getCustomFieldObject(it).name] }
   
// read the grid data
Class dataManagerClass = pluginAccessor.getClassLoader().findClass("com.idalko.jira.plugins.igrid.api.data.TGEGridTableDataManager");
def tgeGridDataManager = ComponentAccessor.getOSGiComponentInstanceOfType(dataManagerClass);
 
// get current user
JiraAuthenticationContext jiraAuthenticationContext = ComponentAccessor.getOSGiComponentInstanceOfType(JiraAuthenticationContext.class);
Object userObject = jiraAuthenticationContext.getLoggedInUser();
User user = userObject instanceof ApplicationUser ? ((ApplicationUser) userObject).getDirectoryUser() : (User) userObject
 
StringBuilder responses = new StringBuilder()
 
def searchService = ComponentAccessor.getComponent(SearchService)
def loggedInUser = ComponentAccessor.getJiraAuthenticationContext().getLoggedInUser()
def queryParser = ComponentAccessor.getComponent(JqlQueryParser)
 
//your query goes here
def query = queryParser.parseQuery(jqlQuery)
 
//gets results of query
def search = searchService.search(loggedInUser, query, PagerFilter.getUnlimitedFilter())
 
//iterate over each returned issue
Collection<Issue> issues = search.results
     
// Cloud URL
def baseURL =  "https://databuckets.net/rest/tgc/api/v1/grids/"
 
// Get all Cloud Grids
 
def jsonSlurperAccount = new JsonSlurper()
URL getGridsURL;
getGridsURL = new URL(baseURL);
URLConnection getGridsConnection = getGridsURL.openConnection();
getGridsConnection.setRequestMethod("GET")
getGridsConnection.doOutput = true
getGridsConnection.setRequestProperty("Authorization", authorizationStringCloud)
getGridsConnection.connect();
 
def getGridsResponseCode = getGridsConnection.getResponseCode()
def gridsList
 
if(getGridsResponseCode != 200){
    responses.append("Error while getting the list of Grids in Cloud")
}else{
    gridsList = jsonSlurperAccount.parseText(getGridsConnection.getInputStream().getText());
}
 
Map<Long,String> gridCorrespondance = [:]
 
for(grid in tgeCustomFieldNames){
    if(gridsList.find { it.name == grid.values()[0].toString() } != null) {
        gridCorrespondance.putAll([(grid.keySet()[0]): (gridsList.find { it.name == grid.values()[0].toString() }.id)])
    }
}
 
for (Issue issue : issues){
     
    for (Long tgeCustomFieldId : tgeCustomFieldIds) {
 
        String issueKey = issue.getKey()
        def issueIdServer = issue.getId()
 
        def gridIdCloud = gridCorrespondance[tgeCustomFieldId]
     
        def baseURL2 =  baseURL + gridIdCloud + "/issue/" + issueKey
 
 
        try {
            def callResult = tgeGridDataManager.readGridData(issue.getId(), tgeCustomFieldId, null, null, 0, 10, user);
 
            def serverDataObject = callResult.getValues()
 
            // responses.append("Grid ID=" + tgeCustomFieldId + " content: " + callResult.getValues() + "\n");
 
     
            def rowsToAdd = []
 
            if(serverDataObject.size > 0){
                 
 
                for(row in serverDataObject){
                    rowsToAdd.add(JsonOutput.toJson(row.collectEntries { [it.key.toString(), (it.value instanceof HashMap) ? it.value.name : it.value] }))
                }
 
                // Generating the JSON body to pass onto the Cloud API and removing all irrelevant data
 
                def cloudJsonObject = jsonSlurperAccount.parseText( '{ "rows":' + rowsToAdd.toString() + '}')
 
                cloudJsonObject.rows.each { e ->
 
                    ArrayList<Object> toRemove = new ArrayList<>();
 
                    e.remove("issueId")
                    e.remove("modified")
                    e.remove("id")
 
                    e.each{ key, value ->
                        if(!value || value === null || value === 'null'){
                            toRemove.add(key)
                        }
                    }
 
                    for (o in toRemove) {
                        e.remove((String)o);
                    }
 
                }
                 
                 
                // Geting Grid info from Cloud to see if it was already uploaded (avoid duplicates)
 
                def gridRowsinCloud = -1
 
                try {
 
                    URL url3;
                    url3 = new URL(baseURL2);
                    URLConnection connection3 = url3.openConnection();
                    connection3.setRequestMethod("GET")
                    connection3.doOutput = true
                    connection3.setRequestProperty("Authorization", authorizationStringCloud)
                    connection3.connect();
 
                    responseCode = connection3.getResponseCode()
 
                    if(responseCode != 200){
                        responseMessage = connection3.getErrorStream().text;
                    }else{
                        String cloudData = connection3.getInputStream().getText();
                        def cloudjson = jsonSlurperAccount.parseText(cloudData)
                        //Long gridRowsinServer = object.rows.size();
                        gridRowsinCloud = cloudjson.rows.size(); //if it works, change value of variable
                    }
                } catch(java.io.IOException ex) {
                    responses.append("!! exception !!");
                }
 
 
                // Uploading the grid data to the Cloud if the target Grid is empty (non-empty grids are assumed to have been already migrated)
 
                if(gridRowsinCloud <= 0){
 
                    try {
 
                        URL url2;
                        url2 = new URL(baseURL2);
                        URLConnection connection2 = url2.openConnection();
                        connection2.setRequestMethod("POST")
                        connection2.doOutput = true
                        connection2.setRequestProperty("Authorization", authorizationStringCloud)
                        connection2.setRequestProperty("Content-Type", "application/json;charset=UTF-8")
                        connection2.outputStream.withWriter("UTF-8") { new StreamingJsonBuilder(it, cloudJsonObject) }
                        connection2.connect();
 
                        responseCode = connection2.getResponseCode()
 
                        if(responseCode != 201){
                            responseMessage = connection2.getErrorStream().text + JsonOutput.toJson(cloudJsonObject);
                        }else{
                            responseMessage = connection2.getResponseMessage();
                        }
 
                        responses.append(   "  ||  IssueKey: " + issueKey +
                                            " ResponseCode:" + responseCode +
                                            " ResponseMessage:" + responseMessage)
 
                    } catch(java.io.IOException ex) {
 
                        responses.append("!! exception !!");
                     
                    }
 
                    responses.append("\n")
                }
            }
 
            } catch (Exception e) {
 
            responses.append("Grid ID=" + tgeCustomFieldId + " data cannot be retrieved: " + e + "\n");
        }
    }
}
 
return responses

Prerequisites

  • A plugin/tool for running Groovy code is present on the instance from which the script is being run

  • TGE app is installed and enabled on Server/Datacenter, TGNG Cloud is installed and enabled on Cloud

  • The project from the source instance was already migrated to the target instance

  • All Columns in the Grid Config are made editable before running the script (can be changed after)

  • Please note that the script was written to be used as a sample and requires you to change the variables' values

  • None of the Target Grids have default rows (they can be re-added after the data is migrated)

Disclaimers

  • The script was developed to be run from the source Server/Datacenter instance.

  • The script is not an official solution for migration but it’s a workaround solution and should be used with care and precaution.

Guide

  1. Make sure that all of the prerequisites above are met

  2. Verify whether the grid fields on your Server instance have a multi-context setup

    1. If not, configure identical Grids on the Cloud instance (+ Scopes)

    2. If so, create an identical Grid configuration for each distinct context

  3. Obtain a valid authentication string from the Cloud instance and replace the placeholder value in the script with it

  4. Prepare the script for verification by adding a logic gate in the "for" loop or by setting a JQL string so that it only returns a single issue

  5. Verify whether object.rows.each { } loop does not contain any mentions of columns that are not present in the Grid configuration and add e.remove("COLUMN_NAME") for all sequence columns

  6. Run the script and verify data, for each Grid or Grid context

  7. After the finalised script has been validated on one of a few (up to 10) issues, proceed to remove the logic gate added in step 4 or expand the JQL string to an expression covering all relevant issues

  • No labels