Versions Compared

Version Old Version 6 New Version 7
Changes made by

Hung Techlead

Hung Techlead

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

  1. The user defines an SQL data source

  2. In the Grid config, the user sets up Data Mirror and checks the box to make it the primary storage

    1. An info panel with table handling restrictions is displayed (Do not change the table schema or it will break)

    2. A checkbox is presented labelled something like “Treat DataSource value as primary during sync (overwrites Jira value)”

  3. Any pre-existing data is mirrored to the DB Table (as in Data Mirror)

  4. Any changes in the DB Table after that are reflected on Issue View and when REST API is called

    1. If an error occurs because of DB Table data type incompatibility (or any other SQL-related data), the error message should include that information

  5. Any changes by the user (on issue or via REST/Java API) are first reflected in the table and then saved to the default Jira storage

    1. If the save is not possible due to the change in the DB Table, an error should be displayed detailing the cause

  6. The JQL search is using the most up-to-date values that are taken from the DB Table

Use-case clarification

Usecase

Description

Questions to Clarify

Answers

1

Connect to the datasource

Users configure the connection from the app to the external datasource.

  1. What type of datasource is it? (SQL, Posgres, Oracle, MSSQL)

  2. Which source will be the primary? Database or App?

  1. Support all of them

  2. Suggest “Database” to be the main source

2

Display data from the datasource

Data from the datasource is displayed in tables within the app.

  1. Is there a limit to the number of records displayed?

  2. How to handle with big data from database?

  1. Status
    colourBlue
    titleTBD

  2. Status
    colourBlue
    titleTBD

3

Update data in the app

Users can update data in the app, and the changes are reflected in the external datasource.

  1. How should conflicts between app and datasource data be resolved?

  2. How can we detect the data as conflict? Should users be notified about unsynced data? Should a warning display for stale data?

  3. What happen when the connection is lost? Can still user be able to use that grid?

  4. How should conflicts be handled if datasource changes during editing?

  1. If the data is not the same, the grid will still show the data (which is saved from Jira database). But in this case, user will not know if the data is not synchonize.

  2. We need to find a way to let user know if the data is not the same…

  3. When the connection is lost, the user can still use the grid, but the data will no longer sync with the database.

  4. The action that is saved last will determine the final data value.

4

Update data in the datasource

Updates made in the datasource are synchronized back to the app.

  1. Is real-time synchronization required, or will periodic updates suffice, or when user reload the grid?

  2. How will the app handle partial updates or missing fields?

  3. What happens if a record is updated in the app and datasource simultaneously?

  4. Should updates from the datasource queue when the user is offline and apply later?

  1. Realtime is needed, and user can synchonize again when click on a button “Re-synchonize”.

  2. Follow TGE, with the column is not the same type or missing, it will be ignored.

  3. The action that is saved last will determine the final data value.

  4. Status
    colourBlue
    titleTBD
    A background function to run the synchonize is an option, but not sure if we should support this or not…

5

Synchronize datasource schema

Automatically detect and adapt to the structure (schema) of the datasource for compatibility.

  1. Should the schema be fetched dynamically or provided manually?

  2. How will changes in the datasource schema be handled?

  3. Should users be notified if new columns or tables are added to the datasource?

  4. How to handle if app and database have different restriction? Like allow null value, different in data type?

  1. Status
    colourBlue
    titleTBD
    not sure if we supoprt this

  2. Status
    colourBlue
    titleTBD
    not sure if we support this

  3. Status
    colourBlue
    titleTBD

  4. Showing error when user saves data in the app is the option I suggest.

6

Error handling and notifications

Users are informed of connection issues, synchronization errors, or invalid data.

  1. What types of errors should trigger notifications?

  2. Should there be retry mechanisms for failed synchonize or connections?

  3. What level of detail should be provided in error messages?

  1. Status
    colourBlue
    titleTBD
    Errors related to the table structure not the same, synchonize action failed

  2. Yes, we should provide a button for users to proactively synchronize. However, we will notify them that all current data will be overwritten by the data from the database. Since the synchronization process might take some time, we need to warn users not to interact with the grid during this period. Additionally, consider providing a progress indicator to enhance the user experience.

  3. Status
    colourBlue
    titleTBD
    Show them clearly which column type is preventing user from saving the data.

7

Security and access control

Ensure that only authorized users can access and update data.

  1. What roles/permissions are needed for accessing and updating data?

  2. Is encryption required for data in transit and at rest?

  3. Should users' actions (e.g., edits, deletions) be restricted based on their roles?

  1. User that can edit grid can do this, or admin can use the Grid Behaviour to restrict

  2. Status
    colourBlue
    titleTBD

  3. The same as #1 answer

8

Logging and audit trail

Log all actions related to data synchronization for troubleshooting and auditing purposes.

  1. What details should be logged (e.g., timestamp, user, action)?

  2. Who should have access to the logs?

  3. Should there be a UI for viewing logs within the app?

  1. Status
    colourBlue
    titleTBD
    Something the same as Grid History should be logged.

  2. Admin

  3. Yes, but in the Grid Configuration → Data Mirror section for only admin

9

Data transformation and mapping

Define how the datasource data is mapped to the app's data structure.

  1. What if the column type is being changed from one side, what should be happen?

  2. How will unsupported or invalid data be handled?

  1. With TGE, it does not support changing column types and reflecting those changes in the database and vice versa, if the changes is being processed, the data in table will be broken and does not display the data. TGNG should have the same restriction.

  2. Ignore and only work with correct data

Plan of action:

  1. Check how TGE handles a similar functionality

  2. Implement a naїve version of the functionality (always overwrites on sync, doesn’t handle getCustomfieldValue, etc.)

  3. Validate whether all of the points in the Questions are relevant to the actual behaviour of the Grid in that basic implementation version.

  4. Write potential answers/suggestions to the questions with some detail on the implementation.

  5. Get approval on the clarifications from Questions

  6. Agree and document the timescale of the full implementation, and add it to the overall roadmap.

  7. Implement a more thorough version of the functionality

Questions:

Which source will be the primary? Database or App?

Question

1

Suggest database should be the primary source due to it security

2

How often does data sync? Real-time, period, or when user click on “Re-synchonize”?

3

How to recognize data loss of sync?

4

How the temporarily data from App will be handled? When user change data from App but lost connection to database, these changes will be saved to which place?

5

How the UI interact when the synchonize is being process?

6

If the connection to the DB is lost temporarily, do we lock the Grid from being edited? Do we block it from being viewed?

If the Grid remains editable, do we sync the data after the connection with the DB is restored? How is that handled given that the normal behaviour in this case is that DB data overrides Jira data, since DB is the primary storage?

Hung: To simplify the process, we can display a warning to notify users when the data is not synchronized correctly. Additionally, we can provide a "Re-synchronize" button, which, when clicked, will fetch all the data from the database and replace the current data in the UI grid. This approach ensures that the database acts as the primary source for synchronization.

7

How can we save the history of changes?

8

How to recognize and display error to the app when the connection is lost?

9

When the connection is lost, can user still use the grid? What will happen in the App in this case?

10

What happen when many users update a grid?

11

Should the data between app and database be encrypted?

12

How can we handle with big data from database?

13

How to handle if app and database have different restriction? Like allow null value, different in data type?

142

Does this functionality need to be present on Cloud as well as on Server

Hung: There is no restriction so I think we can do for both

15

Support Offline?

16

Process for user to track the synchonize action?

173

How the TGE manages the getCustomfieldValue call from Jira’s own API? Do we provide the data pulled from the table on that call? Is it a Service that TGE exports?

18

Will we support other database type beside SQL? Like Postgresql, MSSQL, Oracle?

194

How will the Row ID be handled when new rows are added on the DB side and not through the regular means (manual, API)? Does the ID get generated when the data is loaded into the Jira Grid view? Does it get generated automatically?

Does the DB data overwrite default rows automatically on issue-create if the user enters a non-existent Issue ID first and then an Issue with that ID is created?

Hung: We keep auto_increment ID on both side, and use another field “UUID” to manage the data to synchonize.

With the data save from FE, we can use library like uuidv4

WIth the data save from database, we can use function UUID() from mysql

Code Block
CREATE TABLE MyTable (
    id INT AUTO_INCREMENT PRIMARY KEY,
    uuid CHAR(36) NOT NULL DEFAULT (UUID()),
    name VARCHAR(255),
    age INT
);
205

If the DB Table sourced rows contain non-standard Row IDs, do they get transformed and re-inserted? Can we make the correct format of the Row ID be populated on the SQL side?

Hung: the same answer as above

216

Should we use a different format for Row IDs that are primarily stored in a remote DB?

Hung: UUID is an option that I suggest

227

Should the case of initialising TGNG from a pre-existing table be supported?

Hung: Yes, that is also a case that Francis suggest when we talk about bulk import

238

Do we handle Attachment columns by just storing the Attachment ID?

HUng: Tested on current Data Mirror function, it will save the Attachment ID with the value property

Screenshot 2025-01-07 at 16.40.18.png
249

Do wrong/non-existing Attachment ID values get ignored or do we add a custom error? Does the same behaviour apply to wrong/non-existing values on User/Multi/Single-select?

Hung: With the current behaviour, it will ignore, so do User/Multi/Single-select. But to make it more effective, we can display a #error like Excel