
Issues to think about and address in the future:

--------------
Jan 31, 2019:


-- Minor mistakes and tough to notice errors in the user function help can mean it's not loaded and parsed properly in the registry. Need to fix this. Maybe add a tool where you enter {function name, input names, output names, parameter names} and it generates help text in the hmr*.m file. 

-- DONE: Add default traversal behavior for files in the Homer3 listbox so that user doesn't have to contantly click on the group, run, subj radiobuttons - include in this default behavior for datatype radiobuttons buttons. 

-- DONE: Add ability to fix x and y ranges in the homer3 data display. 

-- DONE: Rework stimGUI change name to StimEditGUI, get rid of associated class and add it to the ChildGuiClass object hmr.childguis. Do same for PlotProbeGUI and ProcStreamOptionsGUI. These all should just be 
front-end client applications of DataTreeClass and FuncRegClass obects. 

-- DONE: Revisit the idea of making a uniform sizes condition sets between Subj, Group and Run to allow getting rid of the need for CondNames2* tables. It would really, really simplify things with conditions and make the group and subject level proc stream averaging much easier. 

-- Fix the procStream fixing code that doesn't work. Also really need to simplify it and make it part of a class. 

-- Improve performance of various components of Homer3 
a) starting with the facts that it keeps all the data in memory and doesn't need to 
b) variable preallocation in for loops 
c) DONE: ProcStreamEditGUI takes a long time to start and it's responce time is slow.
d) Snirf HDF5 saving/loading 


=====================================================
Feb 1, 2019


-- DONE: Make procStream.input.param part of the FuncClass. It is now part of the ProcInputClass which doesn't make sense and is annoying to work with.

-- DONE: Homer3 should do a strict check at startup time of any group folder it opens of the groupResults.mat and proccessOpt_default.cfg to make sure they are in the Homer3 format. If not either ignore them or have a well defined process for converting from a legacy version to homer3 version. If that's not possible treat these files as if they were not there, move them to different names or delete them altogether. Another option might be to use names for these that are unique to homer3. 

-- DONE: Change unit tests to be entirely automated, to run without any user interaction, so it would ask to load config file. Maybe change the loading process to check for a master config file which Homer3 could read in default choices. 

-- Add another unit tests which makes all sub unit tests fail by automatically modifying one of the source code files in a subtle ways, then unmodifying so that you don't accidentally commit the bad change. 

-- DONE: The unit test that tests different outputs for different values was revealed to be not valid, since faulty homer3 code that produces bad output will also produce a passing results.  Instead you have to have the correct result to compare against with non-default lpf values. Need to fix this. 

=====================================================
Feb 2, 2019


-- Create a Simple basic LoggerClass to generate logging information for every Homer3 session, with version number, time stamps, etc. You should also be able to email the log as an attachment using a menu option. 

-- DONE: Insert Homer3 version numbers into groupResults (I think groupResults might already have it, check to make sure) and any proc stream config file, processOpt_*.cfg. 

-- DONE: DataFilesClass should use the same (acquisition-level) classes as the DataTreeClass to load and vet the data files. Also Homer3 should load the files ONLY ONCE not twice like it's done now. This will not only speed up the launching process but also make it much more coherent where the data files use the same mechanism to check their validity as the DataTreeClass object. 

=====================================================
Mar 1, 2019


-- DONE: Come up with a modular object oriented way to update Homer3 display after changes in stimGUI. Right now it does not update unless you do it manually - that is after changing stimGUI you have to click on a control in Homer3 to see the stim changes in Homer3 display.

-- DONE: Come up with a modular object oriented way to update Homer3 listboxFiles control when calculating proc stream. Right now it does not update. 

=====================================================
Mar 12, 2019


-- DONE: Display the group folder as a directory tree in listbox files same as subject folders are displyed for consistency and coherence when determining default behavior of processing level and plot data type button selections.

-- DONE: Use usage names + function names instead of just function names in the ProcStreamOptionsGUI to let user see the proc stream as sequence of function calls (which is what it is) rather than sequence of function names.

-- Add standard deviation calculation to GroupClass.Calc() method.

-- Add checks of dodAvg and dodAvg and dcAvg standard deviations to unit test. 

=====================================================
Apr 10, 2019


-- Fix PlotProbe using different Y scales for multiple data blocks

-- Fix extended channels selection (i.e. waterfall display) not working for multiple data blocks. That is, the colors of the data curves plots in the data axes do not match the selected channel colors. 

-- DONE: When editing and saving current processing stream it's nice to have the ProcStreamOptionsEditGUI updating automatically if it is active at the time of the proc stream update. 

-- DONE: Add to data axes Fix y-range and Fix x-range features that are in Homer2_UI. These are very useful. 

=====================================================
Apr 16, 2019


-- Check the at the channel dataTypeLabels that are being used conform to spec. 

-- Instead of using hard coded numbers for dataTypeLabels to label channels, create a list of data type values in SnirfClass. 

=====================================================
Apr 21, 2019


-- Generate dc and dod timecourses on the fly using active proc stream function when starting Homer3 to avoid having to save this in groupResults.mat. In order to save costly memory usage. 


=====================================================
Nov 6, 2019

Notes from fnirs course 2019

-- Homer3 gets matlab exception in folder which has no write permissions. Need to have a workaround for this. 
-- Label units in MainGUI data axes window
-- Add units to SDG axes 
-- Generate SD file from Homer3


=====================================================
May 11, 2020

-- When loading and rejecting outdated .snirf files, before asking the user about loading a different group folder check for presense of .nirs files and if they're present, do the cycle of telling user about the .nirs files and if they want to convert them to .snirf



-- 1.18.8  vs  1.18.5

-- Before these changes if the file wasn't successfully saved for whatever reason, it was left in a bad state and corrupted state. And next time you repoen H3 it tries to load this file but because it's bad it can;t read it and throws an error. Also another quirk Ive seen is that if it got to that point where a a bad file was created, until you exit matlab you can't even delete the bad file. 

-- Added check 2 things  




Issue 1:

    Since we made a release and since then have made changes to the spec and data structure names and functionality we should have a new version number, no? 
    This matter greatly for instance Homer3 where people are getting errors as I make changes to Snirf functionality to match latest spec. I want to be able to 
    compare currently supported version with the loaded snirf file's version and based on that know what to do: warn user that loaded file is older version, 
    call an older version's snirf reader/writer, etc.
    
    I propose changing the spec as soon as possible to have formatVersion = 1.1 (or whatever we agree on) and have that be a new release 

    I changed the latest Homer3 to produce snirf files with formatVersion == '1.1', in order to be able to compare to formatVersion in existing snirf files. 

Issue 2:

    We (Qianqian and I) have /nirs/probe/frequency not matching spec which says it should be named /nirs/probe/frequencies. I changed this in my snirf reader/writer
    
    
Issue 2:  

    Generating 2D arrays for data items specified as 1D arrays. Examples include /nirs/probe/frequencies and /nirs/probe/wavelengths.

    easyh5-generated  sample when read in python produces a 2D array which does not agree with spec:

        >> h5disp('./Simple_probe.snirf')
        Dataset 'frequency' 
            Size:  1x1
            MaxSize:  1x1
            Datatype:   H5T_IEEE_F64LE (double)
            ChunkSize:  []
            Filters:  none
            FillValue:  0.000000

    Homer3 generated sample when read in python produces 1D array according to spec:
    
        >> h5disp('./Simple_probe.snirf')
        Dataset 'frequency' 
            Size:  1
            MaxSize:  1
            Datatype:   H5T_IEEE_F64LE (double)
            ChunkSize:  1
            Filters:  none
            FillValue:  0.000000


Issue 4: 

    While string arrays are bit of a gray area as far as the spec is concerned, I was curious how the various softwares would save and load them in python where storage order is not an issue relative to HDF5.  As Qianqian has suggested a row-major language is a good way to independently verify your snirf files. So I tried to read the string-array data items in a python interpreter and it seems to show homer3 handling string arrays for sourceLabels and detectorLabels for instance, better than easyh5.

    easyh5-generated

        C:\Users\same1\workspaces\snirf-samples\basic>python
          . . . . . . .       
        >>> import numpy as np
        >>> import h5py
        >>> fid = h5py.File('Simple_probe.snirf','r')
        >>> sl = np.array(fid.get('/nirs/probe/sourceLabels'))
        >>> sl
        array([[b'S', b'1']], dtype='|S1')
        >>> dl = np.array(fid.get('/nirs/probe/detectorLabels'))
        >>> dl
        array([[b'D', b'D', b'D', b'D'],
               [b'1', b'2', b'3', b'4']], dtype='|S1')



    snirf_homer3-generated
    
        C:\Users\same1\workspaces\Homer3\DataTree\AcquiredData\Snirf\Examples>python
          . . . . . . .       
        >>> fid = h5py.File('Simple_probe.snirf','r')
        >>> sl = np.array(fid.get('/nirs/probe/sourceLabels'))
        >>> sl
        array([b'S1'], dtype='|S2')
        >>> dl = np.array(fid.get('/nirs/probe/detectorLabels'))
        >>> dl
        array([b'D1', b'D2', b'D3', b'D4'], dtype='|S2')


Also I added a python SNIRF reader (SnirfClass.py) to Homer3 and snirf-homer3. It reads a snirf file and summarizes it's contents in a printout. The attachements contain a full summary of equivalent file printouts. 


=============================
Jun 17, 2020

Zoom Meeting with Steve Tucker, David Boas and 
-- Ask about version number checking. Am I supposed to change to v1.0?
-- Snirf validator, Snirf samples git repo, HDFView, Data space. 
 

=============================
Jul 17, 2020


groups = '/g1/g2/g3/g4/g5/ds'
groups = fileparts(groups)
r = fileparts('/')
r = fileparts('/g1')
groups = '/g1/g2/g3/g4/g5/ds'
err = HDF5WriteDataset2('tryme.h5', groups, data)
delete tryme.crap
delete tryme.*
err = HDF5WriteDataset2('tryme.h5', groups, data)
delete tryme.*
err = HDF5WriteDataset2('tryme.h5', groups, data)
fid = H5F.create('tryme.h5', 'H5F_ACC_TRUNC', 'H5P_DEFAULT', 'H5P_DEFAULT');
gid = H5G.open(fid, '/')
H5G.close()
H5G.close(gid)
H5F.close(fid)
delete tryme.*
delete tryme.*; err = HDF5WriteDataset2('tryme.h5', groups, data)
dbquit
delete tryme.*; err = HDF5WriteDataset2('tryme.h5', groups, data)
location = groups;
data2 = h5read(fname, location);
fname = 'tryme.h5'
data2 = h5read(fname, location);
data2
data
data3 = round(100*rand(5,3));
err = HDF5WriteDataset2('tryme.h5', groups, data)
data3
data2 = h5read(fname, location);
data2
data
err = HDF5WriteDataset2('tryme.h5', groups, data3)
err = HDF5WriteDataset('tryme.h5', groups, data3)
h5disp(fname)
err = HDF5_DatasetWrite('tryme.h5', groups, data3)
h5disp(fname)
err = HDF5_DatasetWrite('tryme.h5', groups, data3)
H5D.set_extent(dset_id, size(data))


https://www.mathworks.com/matlabcentral/answers/415137-how-do-i-replace-a-dataset-in-an-hdf5-file-with-a-smaller-dataset-without-leaving-some-values-of-th
https://portal.hdfgroup.org/display/HDF5/H5D_SET_EXTENT




==================================
Oct 9, 2020

Christian found issue when opening Homer3 in random folder that it cannot tell if its a valid group folder or group of groups folder and unconditionally 
generates a groupResults.mat file. We have to add feature where Homer3 decides if a group folder is valid. A group folder is valid if it has acquistion files
that are consiStent with each other in terms of having the same probe. 


==================================
Jul 22, 2021

Fixes need to made 

1. configSettingsGUI bug
2. Error in plot ptobe with synch browsing, close exported fig, change curr element
3. Keep all exported figures in plot probe open instead of having one exported figure 
4. Make sync browsing default in plot probe

