Article de blogue en français.
An updated version of the Data Explorer for Odesi and Borealis is now available!
Following the release of the new and improved Data Explorer earlier this year, we have continued to make improvements to the tool’s features and interface over the past few months. This updated version of the Data Explorer includes bug fixes and improvements for variable selection and groups, cross tabulations, bulk editing, applying weights, importing DDI XML files for reuse, and generally increased speed and performance.
An overview of the updated features and fixes for the Data Explorer is provided below. For more resources on getting started with the Data Explorer, please refer to the Odesi User Guide.
- Variable selection & groups
- Cross tabulations
- XML import and file uploads
- Bulk editing
- Speed, performance, and UI
- Contact us
- About Data Explorer
Variable selection & groups
Previously, some curators had issues creating and adding to variable groups, noting that once a variable group had been created and they attempted to assign a variable to that group through the ‘edit variable’ option, they would receive an error message and were unable to save their changes back to Dataverse. Some users also noted occasional duplication when selecting variables to add to groups, as well as lag times for variable selection in some browsers, specifically Chrome. These issues have now been addressed and users are able to add and remove variables from groups with no error message, with increased speed for selecting variables, and without duplication.
Additional updates to variable features include automatic sorting of variables alphabetically, as well as a new option for users with admin permissions to remove weights from multiple variables at once using the “select” menu when curating datasets.
Cross tabulations
Following reports from some users of issues with weight applications in the cross tabulation feature, cross tabulations and charts will now automatically update when weights are applied to the selected variables.
Please note that the team is currently fixing the "Show Value" options to display percentages in addition to counts in the table view.
Additionally, the “Categories (Filter)” option in the Cross Tabulation builder now shows all categories applicable to the selected variable(s) as automatically checked to be included in the table. Previously, the default was to include all applicable categories and users would need to select a given category from the drop-down list in order to exclude (show as a checked box). In order to make this feature more intuitive, the applicable categories will still be included by default, however this will be shown as all categories selected in the drop-down menu and users may deselect a category to exclude it from the table.
By default, all categories are checked. When users deselect a category in the Filter drop-down, it is automatically removed from the cross tabulation table.
XML import and file uploads
The Data Explorer allows users with curator and admin permissions to reupload DDI XML for metadata reuse when curating datasets within a continuous series (for example, the monthly releases of the Labour Force Survey). Previously, the Data Explorer only allowed an XML file to be uploaded once for any given dataset. Some curators noted use cases where it would be helpful to be able to upload DDI XML metadata for reuse multiple times, and this option is now enabled in the Data Explorer.
Bulk editing
Users previously reported errors with the bulk editing features for variables in the Data Explorer. The bulk editing feature has now been improved to allow for bulk weighting or removal of weights for variables, as well as bulk editing variable information once a weight has already been applied.
In previous versions of the Data Explorer, users making bulk edits to datasets would occasionally encounter issues when attempting to save edits back to Dataverse. When making bulk edits to a dataset, it takes time for the system to process these changes and apply them to the dataset. If a user attempted to save bulk updates to a dataset immediately after applying them - while these updates are still being processed - an error would occur resulting in a corrupted file being uploaded to Dataverse.
In order to make users aware of the processing time needed for bulk edits, the updated version of the Data Explorer now includes notifications to inform the user when the bulk edits to a dataset are being processed, and subsequently when these updates have been applied and can now be saved back to Dataverse. These notifications appear in the top right-hand corner of the screen as “Saving Variable Info” and then “Variable Saved” as confirmation that the user can now proceed.
Speed, performance, and UI
This updated version of the Data Explorer provides generally enhanced performance and speed across all features. However, larger datasets (over 500 MB), such as census files, may take several minutes to load in the Data Explorer due to file size. We are working on adding a notice to the loading page in these cases to warn users of longer than usual loading times.
Recommended Browsers
We have also been made aware of dataset loading problems in chromium-based browsers, specifically Google Chrome and Microsoft Edge. While datasets under 500 MB will load in the Data Explorer using these browsers, datasets over 500 MB will not. This is unfortunately an issue with the browser and not with the Data Explorer itself or Dataverse. While we investigate possible solutions for this issue, we recommend using Firefox or Safari when using the Data Explorer for larger datasets. We have updated the Odesi User Guide and Borealis User Guide to reflect this browser issue and recommendation, and the message added to the “Loading dataset” page in the Data Explorer will include a note to users that Firefox and alternate browsers are recommended for datasets over 500 MB.
Contact us
If you continue to experience issues with the Data Explorer, please contact odesi@scholarsportal.info with information about the dataset in question and specific features in need of troubleshooting, and a member of our team will be in touch as soon as possible.
About Data Explorer
Data Explorer is an open-source web application built and developed by Scholars Portal. The application is fully integrated with the open-source Dataverse repository project developed by IQSS, Harvard University, together with global collaborators. Data Explorer supports users in Odesi and Borealis with viewing and analyzing tabular data files and variables by groups or category, as well as variable metadata, summary frequencies and statistics, building cross-tabulations and charts, subsetting and downloading data.