giseldo/neodataset huggingface dataset:
Investigation of Cypress test improvement

Idea is: how we can be more confident that Geoweb doesn't break.

Result of this ticket is a list of cypress tests to be added, made by @tvanrijn. 
The entire team will review this list, give the priority and then we go forward with the development.

In geoweb-serverless-backend:
- update packages to latest available versions https://gitlab.com/opengeoweb/opengeoweb/-/issues/1146
- add smoketest for using the layer manager
   - login -> open layer manager -> check there is one layer row -> add another layer -> check there is 2 layer rows + 2 legends -> duplicate one layer -> check there is three layer rows + 3 legends -> remove one layer -> check there is 2 layer rows + 2 legends https://gitlab.com/opengeoweb/opengeoweb/-/issues/1147
- add smoketest for using the timeslider (low priority, still under development)
   - login -> open layer manager -> check there is one layer row -> click the timeslider rail -> check that a BE request is made https://gitlab.com/opengeoweb/opengeoweb/-/issues/1148
- add smoketest for selecting a preset https://gitlab.com/opengeoweb/opengeoweb/-/issues/1149
- add smoketest for drag and drop in layer manager (low priority, do once sortable finished) https://gitlab.com/opengeoweb/opengeoweb/-/issues/1152
- add smoketest for drag and drop in TAF (low priority, when implemented again) https://gitlab.com/opengeoweb/opengeoweb/-/issues/1153

In knmi-geoweb FE:
- homepage needs some more unit tests: https://gitlab.com/opengeoweb/opengeoweb/-/issues/1150
![image](/uploads/f14327e96e4c6fd5bffe1fb083c68cf4/image.png)
- all screen manager files need unit tests: https://gitlab.com/opengeoweb/opengeoweb/-/issues/1151
![image](/uploads/85d6a7f52b67e7491ecf95c02227939f/image.png)

Story points: 3.0
FE: Use Final Sortable in Shared and Taf

It has to:
- no errors in strict mode
- it should handle auto overflow
- it should handle auto scroll in list
- use it in sortable first, then in taf. Later in LayerManager
- make sure it works on production build
- should be able to validate fields after (and not lose track of it) with a decent solution (no hacks)


First attempt: dnd-kit. I still in beta works fine, but can't minify it in production. [This is the remove commit](https://gitlab.com/opengeoweb/opengeoweb/-/commit/3aa9d286b6558c795515d75624e3a6bf538a808b)

Seconds attempt: react-beautiful-dnd, worked very simple, only causing issues with validation  
- https://github.com/atlassian/react-beautiful-dnd demo: https://react-beautiful-dnd.netlify.app/?path=/story/super-simple--vertical-list-with-scroll-overflow-scroll
- code sandbox: https://codesandbox.io/s/k260nyxq9v?file=/index.js

![Screen_Recording_2021-09-01_at_14.20.04](/uploads/613a5f5582fb84b2601db7971793c0d2/Screen_Recording_2021-09-01_at_14.20.04.mov)

New attempt:
Look for a suiting alternative with given points in mind

Story points: 8.0
Access to information on keyboard shortcuts from within the GeoWeb application.

MVP mockup by @didier.dezanet: https://www.sketch.com/s/a9c3ae9c-a97e-433c-b8dd-e69d71863abc/a/eKM9Dea/play

The list of keyboard shortcut [here](https://agora.fmi.fi/pages/viewpage.action?spaceKey=GEOWEB&title=User+Guide)

Develop applying the design.

Report the keyboard shortcut in README.

Theming
- make sure to create a component in shared, as it will be used in ref app as well
- make sure it will use correct colors from theme
- use toolcontainer draggable for outer. We should also have a look at the backdrop. Might need a new story for that as well. How does the toolcontainer behave when centering and resize? Check the shared story of the big dialog
- if needed create for Tab component a Theme story first to make sure the correct design is followed color wise. Same for modal behavior

Story points: 5.0
Investigation how to solve the Screen Manager issue

Meeting to understand the issue of using Layer Manager and Legend in all maps in ScreenManager and review of the Store.

The output of the meeting is the creation of Dev Task to solve this issue in Sprint 49.

Evaluate if to put the following task in the sprint (with a good description) or delete completely:
- #1111 
- #1099 
- #1089 

Sander will lead this meeting.

Story points: 3.0
FE: Add back LayerManager to ScreenManager and investigate performance issues

The ConnectedLayerManager gave heavy performance problems in [this commit](https://gitlab.com/opengeoweb/opengeoweb/-/commit/9cc70b3d283e15f98624150ac894da6e41e58d6d) and will be reverted in [this MR](https://gitlab.com/opengeoweb/opengeoweb/-/merge_requests/842). 

We need time to investigate how to fix this issues.

- No props for config should be given
- Outcome: layerManager working like legend currently in ScreenManager

Story points: 8.0
FE: remove named exports of screenmanager

in ScreenManager/index.ts some components and utils are imported and exported. When exported, the names are changed which makes it very unclear how this works. It's fine to name selectors and actions, but for the rest of the component I suggest stick with the names of the components. If the name is not clear, rename the component.

![Screenshot_2021-08-25_at_09.00.27](/uploads/f02827713365fb1b82faa377108d6690/Screenshot_2021-08-25_at_09.00.27.png)

also auto import fix of VsCode does not work with this way with using:
`<ScreenManager.Renderer componentsLookUp={componentsLookUp} />`

Story points: 3.0
FE: theme folder cleanup

Since this lib does not import libs (only MUI and the sortable for table demo), and it will get lots of stories, maybe its' a good idea to make for this an exception; a story folder. Also split the helper functions a bit more, and put themes in a seperate folder. Also start with ReadMe instructions

![Screenshot_2021-08-25_at_08.54.22](/uploads/04a1b3567a7e18b153d98a89f84060bc/Screenshot_2021-08-25_at_08.54.22.png)

Story points: 5.0
FE: fix drag order bug

A fix for the drag order has been fixed here in the reducer (4 files in store/map), it fixes when dragging element 0 to 2 for example mentioned [here](https://gitlab.com/opengeoweb/opengeoweb/-/merge_requests/812#note_652700780).  Seperate it from this MR https://gitlab.com/opengeoweb/opengeoweb/-/issues/1100

Story points: 2.0
FE: Move dynamic module loading outside of the components

Find out if we want 1 dynamic module loader, and if current MapViewConnect is wrong. 


We should load the required modules only once so that there are no duplicates and they don't get reloaded all the time.

- Remove if from Legend
- Only wrap the dynamic module loader on Provider level of store (so on the demo, knmi-geoweb, screenmananger).

Story points: 3.0
FE: TAF fix state update in unmount error

When switching stories in TAF, you can see the following error:
```

react-dom.development.js:88 Warning: Can't perform a React state update on an unmounted component. This is a no-op, but it indicates a memory leak in your application. To fix, cancel all subscriptions and asynchronous tasks in a useEffect cleanup function.
    in TafForm (created by TafFormWrapper)
    in FormProvider (created by ReactHookFormProvider)
    in ReactHookFormProvider (created by TafFormWrapper)
    in TafFormWrapper (created by unboundStoryFn)
    in div (created by ForwardRef(Paper))
    in ForwardRef(Paper) (created by WithStyles(ForwardRef(Paper)))
    in WithStyles(ForwardRef(Paper)) (created by Wrapper)
    in ConfirmationServiceProvider (created by Wrapper)
    in ApiProvider (created by StoryWrapper)
    in ThemeProvider (created by StoryWrapper)
    in StoryWrapper (created by Wrapper)
    in ThemeProvider (created by Wrapper)
    in Wrapper (created by unboundStoryFn)
    in unboundStoryFn
    in ErrorBoundary
```

Story points: 3.0
FE: new legend - remove local state

LegendConnect.tsx now uses a combination of local state and store state to determine if the legend dialog is open or not.

This is due to the fact that if the legendDialog is open and the relevant map is removed, we want to close the dialog so an additional check has been added to determine the final state (stored in the local state).

This local state ensures that the dialog is only opend if:
- We have a valid map
AND
- the store state is also set to opened

To see issue when local state is removed and only relying on global state:
- in screenmanager, open a preset and open the legend
- switch to a different preset. You will see that the legend has now become empty and in the title it shows a non-exitent mapid 

or
- in screenmanager, open a preset with multiple panes and open the legend
- remove the pane from which you just opened the legend
- same problem

Story points: 3.0
BE: empty spaceweather data should result in Grafana alerts

Better monitoring: receiving empty spaceweather data should result in alerts being sent by Grafana to procesbewaker and Slack monitoring channel.

It is necessary to log errors in BE when the data for the incoming streams are empty.
In Grafana the errors alerts are already managed (so nothing is necessary in Grafana at this stage).

In unit test, check if the error log is created!

Story points: 3.0
BE: Kp incoming data monitoring in loop

![image](/uploads/8d70dfde2e192c521983ef82f7f40f55/image.png)

In Grafana (but also in Kibana) the monitoring of KP incoming data (log_type: "datastream" AND product: "kp" for geoweb-lambdas) is in a loop and as shown in the graphs there are multiple counts in less then 1 sec.

It should be only 1 count.

Story points: 3.0
Test/merge fix for SIGMET/AIRMET type in IWXXM30

In issue https://gitlab.com/opengeoweb/opengeoweb/-/issues/1509 the code fixes were done for correct handling of the SIGMET/AIRMET type in IWXXM30.
These changes are in geoweb-knmi-avi-messageservices library version 0.0.21 and this should be tested by deploying a version of the geoweb-serverless-backend with that version.

To check: make different types of SIGMET's and AIRMET's (TEST/OPERATIONAL/EXERCISE) and check if the IWXXM30 output has the correct entries to reflect that status.

Story points: 3.0
FE: make a coreComponentsLookUp that is shared and can be extended

We currently have a ComponentsLookUp in 3 places: knmi-geoweb, geoweb-reference-application and screenmanager storyUtils. That is a lot of duplicate code that could be avoided.

- 1) We should make a core componentsLookup that contains the lookup table for the core components. In the apps it should be possible to extend it with custom components.
- 2) There should be a timeseries componentsLookup in thetimeseries lib.

These can be bundled in the componentsLookup of the geoweb-reference-application.

Keep in mind that we want to have both screenmanager and core libs independent of each other.

Make sure the tests take place in the according library (core components in core).

Story points: 5.0
FE: using @opengeoweb/core package in another project doesn't work anymore

- these imports need to be changed (should be imported from @material-ui/core instead of @material-ui/core/styles etc)
![image](/uploads/c51a47556aa6bce40b5de18bec0ef03c/image.png)
- shared and webmap need to be published to npm (theme is already done in https://gitlab.com/opengeoweb/opengeoweb/-/issues/683)


![image](/uploads/2ecdd48e02e5516a1478f847a85c412f/image.png)

Story points: 3.0
Small fix for IWXXM 3.0 output filename for AIRMET/SIGMET

The filename of the IWXXM 3.0 output for SIGMET/AIRMET contains 2 date/time groups.
Both should contain the start of validity time, according to Jan Koerts.

Uit ICAO Eur doc 33:
```
A Data Switch will route IWXXM data according to the TTAAiiCCCC part of the filename of the File Transfer Body Part. The filename including the current WMO bulletin header will be structured as follows (WMO naming convention A):
A_TTAAiiCCCCYYGGggBBB_C_CCCC_YYYYMMddhhmmss.xml
Where the elements in black and bold are fixed elements and: TTAAiiCCCCYYGGgg is the current WMO header with the date time group BBB is optional (as usual), CCCC is the repeated CCCC part from TTAAiiCCCC, YYYYMMddhhmmss is the date/time group
```
Both date/time groups should contain the validity start of the SIGMET/AIRMET.

Currently the second group contains the publishing date/time, which is incorrect.

Story points: 2.0
BE: logging avi-messageservices customized format

In the geoweb-knmi-avi-messageservices (and related libraries) is used the Slf4j logging method.

Investigation is needed to figure out if it is possible to customize this logger to obtain logs in the following format (that is similar to the format used in geoweb-serverless-backend):

`appType - time - level - class - message`

E.g. AviMessageConverter - 12/DEC/2021:13:14:15 Z - INFO - n.k.g.b.c.S.GeoWebSigmetToFMISigmet - GeoWebSigmetToFMISigmet convert. Error code: xxx. Line: xxx

Specification:
- **appType**: [AviMessageConverter, ...otherLibs]
- **time**: in ISO Time Format
- **level**: [INFO, ERROR, WARNING, DEBUG]
- **class**: the Java class in which the log is triggered
- **message**: the log message should contain a description and the line in code in which it happens.

Part of this ticket is to apply this new format to all logs defined in geoweb-knmi-avi-messageservices (at this moment they are defined as 'log.error','log.info','log.debug','log.warn').
Off course part of this ticket is also to figure out if the current specified logs are useful or not and if are needed more logs.

NOTE: at this moment the current apache logs are in Kibana at this link
https://kibana.dta.logs.knmi.cloud/_plugin/kibana/app/discover#/?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-7d,to:now))&_a=(columns:!(message),filters:!(('$state':(store:appState),meta:(alias:!n,disabled:!f,index:'ecs-apachelog-*',key:docker.container.labels.com_amazonaws_ecs_task-definition-family,negate:!f,params:(query:geoweb-backend-avi-acc),type:phrase),query:(match_phrase:(docker.container.labels.com_amazonaws_ecs_task-definition-family:geoweb-backend-avi-acc)))),index:'ecs-apachelog-*',interval:auto,query:(language:kuery,query:avi),sort:!())

Still open questions:
1. Currently 'trace' logs are specified also (as 'log.trace'). What should we do with those?
2. Can we apply this log format also to other libraries used inside of geoweb-knmi-avi-messageservices, such as fmi-avi-messageconverter, that are slightly out of our control?

Story points: 5.0
FE: fix sigmet-airmet test failing sometimes

see https://gitlab.com/opengeoweb/opengeoweb/-/jobs/1791267839


issue is not fixed, see https://gitlab.com/opengeoweb/opengeoweb/-/jobs/1845843863

issue still not fixed, see https://gitlab.com/opengeoweb/opengeoweb/-/jobs/1858470088

Story points: 2.0
FE: dragging the multiDimensionSelect box is dragging the Map window

- go to knmi-geoweb
- add a layer with elevation, for example Harmonie Temperature PL
- open the dimension select
- try to drag it by clicking the header

expected: dragging only moves the dimension select

actual: dragging moves the entire map window, see video


![DimensionDragging](/uploads/dedfc6a0b3b420b86c7a8adedabb0272/DimensionDragging.gif)

Story points: 5.0
Manage keyword filters in redux store part1

Done after opengeoweb#1475

Add listed keywords to redux store, possible format, where
* isSelected is the state of the checkbox
* keyword as itself works as id

Get all of the keywords from all of the layers from all of the services in redux store.

If select all is enabled, also layers that don't contain any keywords should be shown, way how it could be implemented isn't clear yet. One possible solution is to add a special keyword with id of "All" or something similar, which can be toggled true or false, or detect that none of the keywords have isSelected as false.

```
keywords: [
      {
        "id": "keyword",
        "isSelected": true,
      },
      {
        "id": "keyword"
        "isSelected": true,
      },
      {
        "id": "keyword",
        "isSelected": false,
      },
    ],
```

Story points: 5.0
Create Delete Service Button

Can use branch 1594-Manage-keyword-filters-in-redux-store-part2 as a base, if not yet merged to master:

* Add functionality to delete button in Service Options dialog, shown in picture
  1. Create action/reducer in service store to delete a service
  2. Delete action triggers saga
  3. Saga runs delete service action in LayerSelect store, which is already done by @PauliSairanen

Story points: 2.0
BE: Published TAF not properly superseded - disappears from TAF list

In some cases, a published element is superseded without any new TAF being created. As a result, the TAF for that particular airport and timeslot is no longer available on the TAF list. 

The DB item looks like this:

```
{
 "uuid": "cfcec2db-437d-11ec-983e-67f3fd1e9ec8",
 "creationDate": "2021-11-12T12:00:00Z",
 "taf": {
  "messageType": "ORG",
  "location": "EHAM",
  "validDateStart": "2021-11-12T12:00:00Z",
  "type": null,
  "issueDate": "2021-11-12T09:28:28Z",
  "superseded": true,
  "baseTime": "2021-11-12T12:00:00Z",
  "uuid": "cfcec2db-437d-11ec-983e-67f3fd1e9ec8",
  "validDateEnd": "2021-11-13T18:00:00Z",
  "baseForecast": {
   "valid": {
    "start": "2021-11-12T12:00:00Z",
    "end": "2021-11-13T18:00:00Z"
   },
   "cavOK": true,
   "wind": {
    "speed": 10,
    "unit": "KT",
    "direction": 120
   }
  },
  "status": "PUBLISHED"
 },
 "canbe": [
  "CANCELLED",
  "AMENDED",
  "CORRECTED"
 ]
}
```

![image](/uploads/3f53a53fc486f945d8e24daa3f9844af/image.png)

Looking at the logs, the BE got to this situation when saving a TAF amendment after a failed /taflist call:
![image](/uploads/b291401af28c8402dfc931269415ec31/image.png)

Story points: 5.0
FE: Fix updating dimensions in map on layer change/delete/add

To reproduce:

- Select Harmonie preset in menu, then change one of the layers to a layer with pressure levels. The multi dimension select button for elevation should appear now but it doesn't

Step 1 is to make the redux state map dimensions reflect the current state of the map and not be used as a "memory".

Next step is to try and remember the dimensions but that will be done in a different ticket

Story points: 5.0
FE: Taf has unmounted state error after saving as draft and story

Probably a TAF issue. Easy way for testing is start with the storybook and navigate and observe the errors. 

If you save TAF as draft, this happens:

![Screenshot_2021-11-11_at_08.49.54](/uploads/6e53b1502360849ee6f36e47dfef2525/Screenshot_2021-11-11_at_08.49.54.png)

Als happens when switching stories in http://localhost:4500/?path=/story/components-taf-form--edit-taf-form

Story points: 3.0
FE: Make MultiDimensionSelect also work with 1 dialog and no mapId like we have for legend/layermanager

We need the multidimensionselect to work the same as the legend and layer manager do:
- possible multiple buttons
- all calling the same dialog 

This allows us to use the multidimension select in the screenmanager

Once this has been done - apply to the main page of knmi-geoweb so the multi dimension select is added back in

Should also still work in core without screenmanager

Story points: 5.0
FE: apply theme to screenmanager

Apply correct theme colors so it follows the design (currently the header of the panels for example are always black while they should be a lot lighter for the light theme
- do the theming inside the screenmanager lib so it will look good everywhere. 
- you can probably use the same colors as used in ToolContainer component in the shared lib

Light theme:

![image](/uploads/c80a0412edd1903f9ebb3e9d35363420/image.png)


Dark theme:

![image](/uploads/a9f2f18a49a675fdc5827a86af5a952c/image.png)

Story points: 3.0
FE: Focus to first field after adding a new change group

Focus input on first field after adding. Make sure it behaves unlike the video. 
Some issues found while working on it. 


![Screen_Recording_2021-11-08_at_12.15.04](/uploads/cf7d27cbffdf450a93c4ccbaec94f445/Screen_Recording_2021-11-08_at_12.15.04.mov)

Story points: 3.0
FE: convert presets to screenmanager presets

All the presets from the hamburger menu (see list https://docs.google.com/spreadsheets/d/1xrvkl66oCFj2PU6dprEUiPeLSv8r4qmgYEuO7rfKokk/edit?usp=sharing) need to be added to the screenmanager presets list

Story points: 5.0
FE: replace icons in presets knmi-geoweb

* Replace icons of presets with icons indicating the type of preset (See dev-po channel for some examples from Marcel):

> What could be useful however are icons. Now, we have some icons with the presets in the hamburger menu and it’s obvious that with every preset added, finding the right icon will be increasingly difficult. So, perhaps it’s better to skip the icons from now? That said, a functional icon representing the configuration of the preset (single view, multiple view, single tiled view, multiple tiled views or a kliko) might be very handy indeed!

- Also, make sure to loop the presets, instead of hardcoding every thing. ScreenPresets already have a title, add a viewType prop (singleView,multiview,singleTileview, multipleTileView) and render the prop to an icon.

Story points: 2.0
FE: remove/cleanup presets list in knmi-geoweb

Remove the presets according to this list:
https://docs.google.com/spreadsheets/d/1xrvkl66oCFj2PU6dprEUiPeLSv8r4qmgYEuO7rfKokk/edit?usp=sharing

keep the 2 menu's for now

Story points: 2.0
FE: TAF: updating of TAC in FE is not consistent

- When opening a TAF form, the changegroups are not included in the TAC message
- When focussing, you can see a brief trigger of the loading. The TAC should only change when new data or missing data should show, not between loading

![Screen_Recording_2021-11-09_at_10.05.24](/uploads/abeeb48089d7ee5039195aa80cf88b6a/Screen_Recording_2021-11-09_at_10.05.24.mov)

Story points: 3.0
FE: fix flaky sigmet-airmet test ProductListRow

There is a flaky test in sigmet-airmet making the pipeline fail sometimes see: https://gitlab.com/opengeoweb/opengeoweb/-/jobs/1753344884

Seems to be the following lines:
- (src/lib/components/ProductList/ProductListRow.spec.tsx:298:30)
- (src/lib/components/ProductList/ProductListRow.spec.tsx:316:30)

Story points: 3.0
FE: rename ConfigurableMapWithSlider

In https://gitlab.com/opengeoweb/opengeoweb/-/issues/2186 we've renamed some types and one got removed. Inside the componentsLookUp for core, we render a template based on componentType. Because 1 componentType has been removed in the previous ticket, we have a leftover component called ConfigurableConnectedMap (because now we using both for map with or without timeslider the ConfigurableMapWithSliderConnect component). We can combine these components and rename it:

 
- [x] - put the content of ConfigurableConnectedMap inside ConfigurableMapWithSliderConnect
- [x] - remove ConfigurableConnectedMap (move the tests)
- [x] - rename ConfigurableMapWithSliderConnect to ConfigurableMap 
- [x] - make sure all tests are still valid

Story points: 3.0
FE: update PRESET_BACKEND_URL to tst environment once TST is available

update PRESET_BACKEND_URL to the url of the presets tst environment in:

- `apps/geoweb/src/assets/knmi.config.json `
- `apps/knmi-geoweb/src/assets/config.json `

the url: `gwpresetsbe-tst-master.geoweb.dev.knmi.cloud`

Story points: 1.0
FE: Taf cleanup (story) Provider wrappers

See if we can limit the (story)Providers we use. In presets lib we only use two: one for unconnected components and one for connected, see if we can use that for taf as well.

Check other libraries and create other tickets accordingly. -->
https://gitlab.com/opengeoweb/opengeoweb/-/issues/2212
https://gitlab.com/opengeoweb/opengeoweb/-/issues/2211

Story points: 2.0
FE: presets create snapshot test for MapPresetsMenuItem

Would be nice to snapshot the contents of a MapPresetsMenuItem, since this hold our custom list of actions. For this, the shared ToggleMenu needs also be able to show open on default. 

- add prop to shared ToggleMenu isOpen
- create snapshot test for shared ToggleMenu
- create a new story for MapPresetsMenuItem, and add isOpen to that story and create snapshot test

Story points: 2.0