bobby/open_dbm
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

[New Feature] Add web folder to support web documentation

Browse Source
This commit is contained in:
Rudy Haryanto
2022-10-04 20:17:17 +07:00
parent c27f624ca8
commit a54aa04c9b
251 changed files with 11079 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
docs/website/extras/_fabric-warning.mdx Normal file
View File

@@ -0,0 +1,5 @@
:::caution
This document refers to the architecture of the new renderer, [Fabric](fabric-renderer), that is in active roll-out.
:::

65
docs/website/extras/changelogs-in-pull-requests.md Normal file
View File

@@ -0,0 +1,65 @@
---
title: Changelogs in Pull Requests
---
The changelog entry in your pull request serves as a sort of "tl;dr:" for your changes: do they affect Android? are these breaking changes? is something new being added?
Providing a changelog using a standardized format helps release coordinators write release notes. Please include a changelog as part of your pull request description. Your pull request description will be used as the commit message should the pull request get merged.
### Format
A changelog entry has the following format
```
## Changelog:
[Category] [Type] - Message
```
The "Category" field may be one of:
- **Android**, for changes that affect Android.
- **iOS**, for changes that affect iOS.
- **General**, for changes that do not fit any of the other categories.
- **Internal**, for changes that would not be relevant to developers consuming the release notes.
The "Type" field may be one of:
- **Added**, for new features.
- **Changed**, for changes in existing functionality.
- **Deprecated**, for soon-to-be removed features.
- **Removed**, for now removed features.
- **Fixed**, for any bug fixes.
- **Security**, in case of vulnerabilities.
Finally, the "Message" field may answer "what and why" on a feature level. Use this to briefly tell OpenDBM users about notable changes.
For more detail, see [How do I make a good changelog?](https://keepachangelog.com/en/1.0.0/#how) and [Why keep a changelog?](https://keepachangelog.com/en/1.0.0/#why)
### Examples
- `[General] [Added] - Add snapToOffsets prop to ScrollView component`
- `[General] [Fixed] - Fix various issues in snapToInterval on ScrollView component`
- `[iOS] [Fixed] - Fix crash in RCTImagePicker`
### FAQ
#### What if my pull request contains changes to both Android and JavaScript?
Use the Android category.
#### What if my pull request contains changes to both Android and iOS?
Use the General category if the change is made in a single pull request.
#### What if my pull request contains changes to Android, iOS, and JavaScript?
Use the General category if the change is made in a single pull request.
#### What if...?
Any changelog entry is better than none. If you are unsure if you have picked the right category, use the "message" field to succinctly describe your change.
These entries are used by the [`@rnx-kit/rn-changelog-generator`](https://github.com/microsoft/rnx-kit/tree/main/incubator/rn-changelog-generator) script to build a rough draft, which is then edited by a release coordinator.
Your notes will be used to add your change to the correct location in the final release notes.

98
docs/website/extras/ci-cd-pipeline.md Normal file
View File

@@ -0,0 +1,98 @@
---
id: ci-cd-pipeline
title: OpenDBM CI CD Pipeline
---
## Summary
OpenDBM CI CD Pipeline is very important as its job is to maintain the quality of the library for each new changes. The OpenDBM CI CD pipeline is divided into multiple sections that serve different purposes. Before OpenDBM dive into those sections, please see below the summary of the entirer pipeline flow.
<figure>
<img src="../docs/assets/odbm_pipeline.png" width="500" alt="OpenDBM Pipeline Flow" />
<figcaption>ODBM Pipeline Flow</figcaption>
</figure>
The git flow started when the community or OpenDBM team create stories to do. You can fork from master branch and named the fork based on the requirement (bug fix / enhancement). For instance if its enhancement, the fork name is feature/[github_name]/[feature_name]. The naming convention is not enforced for now but its a good practise to do so. Then after the feature developed and tested locally, the developer need to commit and push their changes into the their forked repository. But in doing so, there will be pre-commit checking which focused on linting will be executed. Upon success, the code will be pushed to your forked repo.
After it pushed, you need to create a pull request. After the PR is created, Github Actions will automatically trigger Code Checking chain processes, to make sure the code is clean and clear of vulnerabilities. It will also run the series of unit tests and run the test cases on different platform (MacOs, Windows, Linux). Only after the Code Checking success, the Release Manager can merge the PR.
After it merged, again the Github Actions will be executed Code Checking on the master branch. After the Code Checking is successful, the developers need to create the tag which marked as release version. When tag created successfully, it will again run the Code Check, and additionally it will run the automatic update of OpenDBM to PyPi.
<figure>
<img src="../docs/assets/odbm_workflows.png" width="500" alt="OpenDBM Workflows" />
<figcaption>ODBM Workflows</figcaption>
</figure>
> The OpenDBM is also using caching to speed up the pipeline deployment. So the installation and dependencies installation will be done in fast time. The only time needed is to download the cache into the github action container. You can check the cache key in job `- uses: actions/cache@v3` in `.github/workflows/open_dbm-code-checking.yml` file.
## OpenDBM Code Checking
The configuration for this pipelines is located under `.github/workflows/open_dbm-code-checking.yml`. Its the pipeline that will trigger in almost every git events. When you push your changes to your forked repository, it will run using your own github actions. It also only triggers when you change the files of the core library such as under dbm_lib directory. Any non-core files changes will not trigger this pipeline.
### Linting
Linting is a way of detecting errors in code that can be solved with syntax or style changes. People often use linting to find simple errors, such as inconsistent spacing or formatting. This is generally more effective than merely running the code.
A good linter will often flag errors that the compiler would not catch. Often these are frivolous problems such as unused variables, but they serve to remind programmers about best practices and are sometimes harder for humans to catch when scanning through a large body of code. In Python, there are many options for linters including PyLint, black and pylama. In ODBM-G2, OpenDBM will use flake8 as our linter.
### Vulnerabilities Scanning
With a 650% increase in next-gen cyberattacks against open source tools recorded in the last year, it is important to secure openDBM code by scanning the possible vulnerabilities from third party libraries.
ODBM-G2 will be using safety (https://pypi.org/project/safety) to scan the third party vulnerabilities.
<figure>
<img src="../docs/assets/odbm_safety.png" width="500" alt="OpenDBM Safety" />
<figcaption>ODBM Safety</figcaption>
</figure>
### Unit tests
Unit Test is a way to test every unit/component in codes. This unit can be a function, method, procedure, or module. Unit tests must be isolated from the original code to verify its correctness.
OpenDBM is using one of the most popular frameworks for unit testing, Pytest. Pytest framework is easier to read, more straightforward to write, and can be scalable to build complex testing.
In OpenDBM project, the unit test will be stored in Test folder. The Unit Test file structure will mimic OpenDBM File Structure to improve the readability / easier to check. For example, as below.
<figure>
<img src="../docs/assets/odbm_unittest.png" width="500" alt="OpenDBM Unit tests" />
<figcaption>ODBM Unit tests</figcaption>
</figure>
To have test case material, OpenDBM use video from various sources to generate all output from ODBM-G1. This video captures all the functionality that OpenDBM has, such as emotional expressivity, eye blink, head movement, speech, etc. The output that is stored in CSV files then are used as our material to do unit testing
There will be 2 types of expected output:
* Non-numeric. If the expected output is non-numeric, then in order to pass the test case, the new codes must generate precisely the same output as the previous codes.
* Numeric. If the expected output is numeric, then in order to pass the test case, the new codes should generate the same output as the previous codes with a margin of error.
Formula for margin of error will be as follow:
tol = max⁡(rel_tol , abs_tol )
With
tol  = Tolerance value between actual output and expected output
rel_tol = relative tolerance based on the magnitude of two numbers
abs_tol= absolute tolerance (constant)
Formula above is a standard for Python [PEP 485](https://peps.python.org/pep-0485/) for testing approximate equality
> Currently the code coverage for OpenDBM is reaching 81%
## OpenDBM Build Checking
This pipeline is to ensure the new changes that will merged to OpenDBM library still supporting one of the great things about OpenDBM, which is supporting all OS platforms. The pipeline not only ensure the installation process run without any problem in all platforms, but also to ensure the functionalities run without problem by executing the test cases in all platforms. If the one of the OS is/are failling due to new changes, it will be notified to the developer and OpenDBM team, and also generate failed badge and display it in the OpenDBM github pages
The pipeline will trigger if there is new Merge Request created.
<figure>
<img src="../docs/assets/odbm_build_checking.png" width="500" alt="OpenDBM Build Checking" />
<figcaption>ODBM Build Checking</figcaption>
</figure>
## OpenDBM Publish Release
This pipeline triggered only if the release manager create a release tag. There is also a condition in the pipeline that the release should not be draft / pre release tag. Only when official release created, it will run the pipeline that lives in `.github/workflows/open_dbm-publish-release.yml`, and publish the new version of OpenDBM library to PyPI
<figure>
<img src="../docs/assets/odbm_pypi_release.png" width="500" alt="OpenDBM PyPI release page" />
<figcaption>ODBM PyPI release page</figcaption>
</figure>
## OpenDBM Web Documentation Deployment
This pipeline is to make deployment of OpenDBM Web documentation. The configuration file is in `.github/workflows/open_dbm-docs-deploy.yml` file. You can find the details about [OpenDBM documentation here](odbm-doc).
> This pipeline only triggered manually. We can go to github actions, select OpenDBM Web Documentation Deployment, the choose the Run WorkFlow
> We need a proper Github Token that can build and push the github pages.
## Github Environment Variables
**GH_TOKEN** : Its to update the badge based on conditions defined in the pipeline. It push the new badge status if there's any status difference after pipeline. Its also used to deploy the Web Documentation to Github pages
**TWINE_PASSWORD** : Pipeline use this variable to push into the PyPI. By default the pipeline use __token__ as the username, indicates that to push to PyPI we will use token instead of credentials

20
docs/website/extras/code-of-conduct.md Normal file
View File

@@ -0,0 +1,20 @@
---
id: code-of-conduct
title: Code of Conduct
---
Alongside the release of OpenDBM v2.0, we decided to release alongside it a code of conduct as is conventional for community-owned repositories. The code of conduct is [included with the codebase](https://github.com/AiCure/open_dbm/blob/master/CODE_OF_CONDUCT.md) when OpenDBM is downloaded but we’ve also pasted it below.
## Commitments
In adopting this Code of Conduct, we are committed to strengthening joint efforts, innovation, and the free exchange of ideas for developing, maintaining, and using open-source software. We hope to build an active and inclusive community for development of methods for measurement of health. We want all members of this community to be active participants regardless of race, gender identity, religion, creed, color, marital status, genetic information, age, nationality, or disability.
## Values
We promote values such as being audience centered, staying calm, professional, and respectful during discussions and communicating through active listening, We encourage all members to respectfully accept evaluation and disapproval, to always aim for what is best for the community, and above all respect their fellow members.
We do not promote any kind of harassment, aggression, or intimidation. Personal threats, attacks, stalking, vulgar language, and any kind of insulting statements will not be tolerated. We do not approve of the exchange of disrespectful language, sketches, or cartoons nor the posting of personal information (emails, etc.) without user permission.
## Reporting
To report any complaint or inappropriate behavior, we ask the users to contact the moderators at [opendbm@aicure.com](mailto:opendbm@aicure.com). Moderators of the community have the right to remove offensive content that does not align with the Code of Conduct. If the content explicitly violates the Code of Conduct, it may result in disqualification from upcoming events and being blocked from the [GitHub Organization](https://help.github.com/articles/blocking-a-user-from-your-organization/)

14
docs/website/extras/community-discussions.md Normal file
View File

@@ -0,0 +1,14 @@
---
id: community-discussions
title: Community Discussions
---
For any and all discussions related to the use of OpenDBM, we encourage all users to host those conversations using the (as of December of 2020) new Discussions feature on Github i.e. the [Discussions](https://github.com/AiCure/open_dbm/discussions) tab on the OpenDBM Github page. While the [Issues](https://github.com/AiCure/open_dbm/issues) tab is for technical issues, we want the Discussions tab to be for everything else. Below are a couple things we want the Discussions tab to be helpful for.
## Discussions on variables
We want to foster threads in the Discussions tab that users can contribute to regarding digital biomarker variables that are clinically relevant measures in the context of individual patient populations. For example, if a user is wondering which of OpenDBM’s outputted variables are relevant to, let’s say, Major Depressive Disorder, we want them to be able to turn to a thread in the Discussions tab where they can find that information. Our team will be starting some of these threads ourselves and adding to them as we run into more literature; we encourage other users to also utilize the Discussions tab to create a central repository of such information.
## Experimental design
One of the more common types of questions we get from users of OpenDBM is about experimental design and data collection setups. As is briefly discussed in Chapter 6, the nature of the data collection paradigm and the behaviors that are relevant differ depending on the disease state and the goals of the experiment. Not to mention––we are not authorities on experimental design whatsoever. If a user has such questions, we encourage them to post them in the [Discussions](https://github.com/AiCure/open_dbm/discussions) tab. We certainly will be monitoring it and responding to any questions to the best of our ability, but we also want to allow other researchers to provide their opinions, which we feel will be just as important to take into consideration.

21
docs/website/extras/contributing-overview.md Normal file
View File

@@ -0,0 +1,21 @@
---
id: overview
title: Contributing Overview
description: How to contribute to OpenDBM
---
<!-- alex disable simple simply -->
Thank you for your interest in contributing to OpenDBM! From commenting on and triaging issues, to reviewing and sending Pull Requests, all contributions are welcome.
We aim to build a vibrant and inclusive [ecosystem of partners, core contributors, and community](https://github.com/AiCure/open_dbm/blob/main/ECOSYSTEM.md) that goes beyond the main OpenDBM GitHub repository.
The [Open Source Guides](https://opensource.guide/) website has a collection of resources for individuals, communities, and companies who want to learn how to run and contribute to an open source project.
Contributors and people new to open source alike will find the following guides especially useful:
- [How to Contribute to Open Source](https://opensource.guide/how-to-contribute/)
- [Building Welcoming Communities](https://opensource.guide/building-community/)
We want to ensure that OpenDBM feels like a community of like-minded researchers and clinicians. Hence, there are a few ways we encourage users to stay involved––and why we [encourage you to join DiME, too!](https://www.dimesociety.org/)
Most importantly, if you’re interested in OpenDBM, star the repo and [sign up for our listserv](https://docs.google.com/forms/d/e/1FAIpQLScLrgyM08YIumzJw8dxmRaMQ4byBL3J4A90FpokDhX8-KUNZw/viewform) for all updates.

16
docs/website/extras/extras.md Normal file
View File

@@ -0,0 +1,16 @@
---
id: extras
title: Resources
---
:::info
Welcome to the Resources section! If you're getting started with OpenDBM, please refer to <a href="/docs/getting-started">Guides</a> section. Continue reading to learn about OpenDBM surrounding components! In this section, we’ve put together some resources that don’t quite fit in any of the other chapters of this documentation but are helpful things to know regardless.
:::
This section is intended to share the surrounding components inside OpenDBM repository. The intended components includes CI/CD pipeline, REST API for demo purpose and how this documentation built. It is not a requirement to be familiar with this material to be effective with OpenDBM. You can still benefit from this section as it will give you insights into how we manage things to give added values to this repository. Feel free to share your feedback on the [discussion group](https://github.com/AiCure/open_dbm/discussions/categories/general) for this section.
## Table of Contents
- [CI CD Pipeline](ci-cd-pipeline)
- [REST API](odbm-rest-api)
- [OpenDBM Documentation](odbm-doc)

5
docs/website/extras/how-to-contribute-code.md Normal file
View File

@@ -0,0 +1,5 @@
---
title: Contribute Guidelines
---
With OpenDBM v2.0, we’ve also written out a simple version of [contribution guidelines,](https://github.com/AiCure/open_dbm/blob/master/CONTRIBUTING.md) available with the code base. We want to make these more detailed in the future. As we work with the first independent contributors to OpenDBM, we will be coming out with a much more robust packet of instructions for contributors. If you find the current guidelines confusing or have trouble contributing to OpenDBM, please definitely [reach out to us](mailto:opendbm@aicure.com) and we’re happy to assist.

6
docs/website/extras/issue-reporting.md Normal file
View File

@@ -0,0 +1,6 @@
---
id: issue-reporting
title: Issue Reporting
---
For any technical issues with using the code i.e. during installation and during use, we ask that you––instead of emailing us––bring up the issue using the Issues feature on Github i.e. in the [Issues](https://github.com/AiCure/open_dbm/issues) tab on the OpenDBM github, We have written out an [Issue template](https://github.com/AiCure/open_dbm/blob/master/.github/ISSUE_TEMPLATE/bug_report.md) that will automatically populate when you create a new issue; we suggest you follow that template.

12
docs/website/extras/license-contact.md Normal file
View File

@@ -0,0 +1,12 @@
---
id: license-contact
title: License and Contact
---
## License
OpenDBM is published under the [AGPL v3.0 License.](https://github.com/AiCure/open_dbm/blob/master/license.txt)
## Contact
For technical issues, we ask that you use the [Issues](https://github.com/AiCure/open_dbm/issues) tab to contact us. For any general discussions, we ask that you please use the [Discussions](https://github.com/AiCure/open_dbm/discussions) tab. If you’d like to contact us privately and directly, you can email us at [opendbm@aicure.com.](mailto:opendbm@aicure.comopendbm@aicure.com)

123
docs/website/extras/odbm-doc.md Normal file
View File

@@ -0,0 +1,123 @@
---
id: odbm-doc
title: OpenDBM Documentation
---
## Summary
As book is the gate to knowledge, OpenDBM documentation is the gate to all the great features provided in this library! The OpenDBM team primary focus is how to make this library easily accessible, installable and usable in a very simple and straightforward as much as possible, and good documentation is the key to our objectives.
## Github documentation
<figure>
<img src="../docs/assets/odbm_github_doc.png" width="500" alt="OpenDBM Github documentation" />
<figcaption>OpenDBM Github documentation</figcaption>
</figure>
We want our github documentation as concise as possible. In here you can find the important information such as:
* The latest OpenDBM version in PyPI
* Unit test results which are coming from OpenDBM Code Checking Github Actions pipeline
* The code coverage that also produced by above process
Along with those informations, you can also see the OS build and test status on our documentation which produced from OpenDBM Build Checking Github Actions pipeline. This way you can check whether the last version of OpenDBM library can support which platforms
### Basic usage and other informations
We also provide a section how to install and use OpenDBM library inside our github page. But of course more detailed explanation are provided in OpenDBM Web documentation which you're probably looking right now
### TOC Table of content
We are using [DocToc](https://github.com/thlorenz/doctoc) in order to generate our table of contents. So dont change the table of content at all. If you want to add anything, just add it like a normal way, and you can use `## My Section` if you feel need to add new section for your information. After that, just generate ```doctoc .``` in the root folder and DocToc will generate the ToC immediately and include your new information seamlessly.
<figure>
<img src="../docs/assets/odbm_doctoc.png" width="500" alt="OpenDBM Github Table of Content Generator" />
<figcaption>OpenDBM Github Table of Content Generator</figcaption>
</figure>
## Web Documentation
By good probability when you read this section, you're currently looking at our web documentation. (Unless you read this from our github blob files which is still normal, no worries!)
> We are using [Docusaurus](https://docusaurus.io/) to generate this documentation as our primary objective is focus on the content and how to deliver the most straightfoward information to the community. So all the neccessary code to build this web documentation are coming from Docusaurus.
## Documentation sections
### Dashboard
When you first arrived to the OpenDBM Web documentation, you will see the concise description about what is OpenDBM and Why to use it. We also provide talks and videos that provide informations about this library. We also put the acknowledgements to those libraries that make everything is possible in this OpenDBM library
### Getting Started
This section provide informations on everything you need to know to install and use the OpenDBM. It explains about the prerequisites before you install the OpenDBM. It makes sure to leave no one behind as we provide the informations to all OS platforms.
<figure>
<img src="../docs/assets/odbm_allOs.png" width="500" alt="OpenDBM Documentation to all platforms" />
<figcaption>OpenDBM Documentation to all platforms</figcaption>
</figure>
### Variables
This section provide deeper informations about OpenDBM. There are a lot of variables you can fetch from OpenDBM, and this section provide you how to use those variables and what is the limitation on each modules. We also make sure to make annotation or reference to acknowledge other people contributions to make all this possible
### API
One of all thing about the OpenDBM generation 2 is how easy it is to use OpenDBM. We put another layer on top of the previous OpenDBM library to make it easier to use OpenDBM. All those APIs informations are provided within this section. This section is automatically generated by [pydoc](https://docs.python.org/3/library/pydoc.html) command line that executed in `.github/workflows/open_dbm-docs-deploy.yml`
<figure>
<img src="../docs/assets/pydoc-pic.png" width="100%" alt="OpenDBM Pydoc Generated Document" />
<figcaption>OpenDBM Pydoc Generated Document</figcaption>
</figure>
### Resources
This section provide all informations surrounding the OpenDBM environment. It gives you information about the pipeline, REST API, Documentation (Hi, there!) and contributing guideines.
### Blog
Within this section, you can read other people perspectives, stories and bunch of interesting stuff about OpenDBM all around the world!
## OpenDBM Web Technical Documentation
Below section will discuss in more detail about the technical aspect of OpenDBM Web documentation structure.
### How to install and run locally
Your node must be set to stable version (as of now version 16) to be able to install and run this documentation
Under the docs directory:
* ```bash yarn``` to install all the dependencies
* Then go to the the `website` directory and run the app by typing:
* Command below is for start the website for the **first time**.
```bash
cd website
pip install pydoc-markdown
pydoc-markdown -I ../opendbm/api_lib/facial_activity -m api --render-toc > website/api/facial-activity-api.md
pydoc-markdown -I ../opendbm/api_lib/movement -m api --render-toc > website/api/movement-api.md
pydoc-markdown -I ../opendbm/api_lib/verbal_acoustics -m api --render-toc > website/api/verbal-acoustics-api.md
pydoc-markdown -I ../opendbm/api_lib/speech -m api --render-toc > website/api/speech-api.md
yarn start
```
* This command is the one you execute if you have already generated dynamic documentation from pydoc-markdown
```bash
cd website && yarn start
```
### Dashboard
The dashboard page is build on top of React framework. You need only a basic React knowledge in order to change stuff in the dashboard.
### Main config file
The main config file of OpenDBM documentation is docusaurus.config.js. You can change things like:
* You can set the footer content
* You can set the `editUrl`. This parameter to define the url when you or other community members want to change this file
<figure>
<img src="../docs/assets/odbm_edit_button.png" width="300" alt="OpenDBM Edit this page button" />
<figcaption>OpenDBM Edit this page button</figcaption>
</figure>
* You can set the url, baseUrl
* in `presets[0][0]` you can set the sidebar file to show in the `getting-started` section. We can also set the limit number of versions that will be displayed in the documentation in this line `['current', ...versions.slice(0, 2)]` We will discuss about versioning deeper below
> Support too many versions will make the web deployment much slower.
* Still in the same above parameter, we can also sync the google analytic tag in `googleAnalytics.trackingID` parameter.
* in `plugins` we also can set the siderbar file to be shown in other section as well
### Versioned pages
This documentation support versioning on the library as it very important that we could provide information and retain it so community can refer to specific version for their usecase. The versions are listed under `docs/website/versions.json`. The latest version must be the first element in that array.
That versions will be referred and used in these two directories `versioned_docs` and `versioned_sidebars`. So you can edit the documentation in specific version within these directories.
Those files and sidebars file also has duplicated contents in `sidebars.json` and files under directories `docs/docs`. These files are meant for future version. So if the community using **Edit This Page** button in the pages, it will create the MR for changes related to this directory. So you can say its a temporary directory for future changes on the documentations. Later, when you create future `version-x.x` version, you can just copy paste all the files under `docs/docs` to the new version directory.
### Non Versioned pages
Right now, the non versioned pages are API, Resources, Blog. It has its own directories under docs/website. If you need new section, you can create the directory and define it in `docusaurus.config.js`.
### Navigation (Sidebar and page ID)
In the `version-xx-sidebars.json`, you can define the sidebar and the IDs of pages that you want to categorize. You can set the id inside the page i.e.
```
id: action-units
title: Actions Units
```

206
docs/website/extras/odbm-rest-api.md Normal file
View File

@@ -0,0 +1,206 @@
---
id: odbm-rest-api
title: OpenDBM REST API
---
import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem'; import constants from '@site/core/TabsConstants';
## Summary
Inside the OpenDBM repository, there is a folder named rest_api. This folder means to showcase that you can leverage OpenDBM library in API service or Web application, which this folder will showcase the former.
> The OpenDBM API designed with OpenAPI 2.0 to standardize API definition for programmatic usage, better developer experience and documentation. It will also protected by standard security protocol.
<figure>
<img src="../docs/assets/odbm_api_summary.png" width="500" alt="OpenDBM REST API Summary" />
<figcaption>ODBM REST API Summary</figcaption>
</figure>
## How to use
To use the Rest API is very straighforward
* Go to the `rest_api` directory
* Install the neccessary dependencies
```commandline
pip install -r requirements.txt
docker pull opendbmteam/dbm-openface
```
* Then you can use Make syntax to run it
```commandline
make dev
```
Wait until the rest api ready to serve requests
```bash
INFO: Will watch for changes in these directories: ['/Users/smarty_not_dummy__user/open_dbm/rest_api/app']
INFO: Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO: Started reloader process [80258] using StatReload
INFO: Started server process [80275]
```
* You can test using the OpenAPI specs like mentioned in [here](ci-cd-pipeline#unit-tests). You can open `http://127.0.0.1:8000/docs` or `http://127.0.0.1:8000/redoc`.
* Before you start using the APIs, you need to create your credentials for authentication. Alternatively, you can try to use username 'aicure' and the password is 'opendbm'.
> You can skip below steps if you want to use 'aicure' credential
* To create your own username, first you generate your own password by following these steps:
```bash
cd rest_api/app
python
Python 3.7.13 (default, Mar 28 2022, 07:24:34)
[Clang 12.0.0 ] :: Anaconda, Inc. on darwin
Type "help", "copyright", "credits" or "license" for more information.
```
Then you can start type your own password using this python command
```python
from services.auth.auth import get_password_hash
print(get_password_hash('test'))
$2b$12$qYnAxjQd..feDyPBYa/mDuRrH.PkcpNHJk3F.l/c0h3l/1NcZup7O
```
Save the password in the last line and your preferred username in app/services/auth/auth.py
```python
def get_db():
fake_users_db = {
"your_new_username": {
"username": "your_new_username",
"full_name": "New Full name",
"email": "new email",
"hashed_password": "$2b$12$qYnAxjQd..feDyPBYa/mDuRrH.PkcpNHJk3F.l/c0h3l/1NcZup7O ",
"disabled": False,
},
"aicure": {
"username": "aicure",
"full_name": "AiCure OpenDBM",
"email": "opendbm@aicure.com",
"hashed_password": "$2b$12$k4R5SPuHkjFKBsQV5gAHl.e/BlxrX2z1H3vxiB9uGtaDZLFXjggCm",
"disabled": False,
},
"alice": {
"username": "alice",
"full_name": "Alice Wonderson",
"email": "alice@aicure.com",
"hashed_password": "fakehashedsecret2",
"disabled": True,
},
}
return fake_users_db
```
* After that you can login using either of these icons, or using the curl alternative
<Tabs
defaultValue="docs"
values={[
{label: 'API Docs', value: 'docs'},
{label: 'cUrl', value: 'curl'},
]}>
<TabItem value="docs">
<figure>
<img src="../docs/assets/opendbm_auth_symbol.png" width="700" alt="OpenDBM Auth" />
<figcaption>ODBM Auth</figcaption>
</figure>
</TabItem>
<TabItem value="curl">
```bash
curl 'http://127.0.0.1:8000/odbm/v1/login' \
-H 'Accept: application/json, text/plain, */*' \
-H 'Authorization: Basic Og==' \
-H 'Content-Type: application/x-www-form-urlencoded' \
--data-raw 'grant_type=password&username=aicure&password=opendbm' \
```
</TabItem>
</Tabs>
* Start with uploading the files, put the "local" string value in the `platform` parameter, then save the upload file name.
<Tabs
defaultValue="docs"
values={[
{label: 'API Docs', value: 'docs'},
{label: 'cUrl', value: 'curl'},
]}>
<TabItem value="docs">
<figure>
<img src="../docs/assets/opendbm_upload_file.png" width="700" alt="OpenDBM File Uploading" />
<figcaption>ODBM File Uploading</figcaption>
</figure>
</TabItem>
<TabItem value="curl">
```bash
curl 'http://127.0.0.1:8000/odbm/v1/upload?file_name=facial_speech_verbal_video_test.mp4&file_extension=mp4&platform=local' \
-H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhaWN1cmUiLCJleHAiOjE2NjI5MDM4MjB9.oYW3z44NdcBLZswxy4Vt1S2r92wYmus59NWnZYDr2CA' \
-H 'Content-Type: multipart/form-data; ' \
-H 'accept: application/json' \
--data-raw $'Content-Disposition: form-data; name="file"; filename="facial_speech_verbal_video_test.mp4" Content-Type: video/mp4 \
--compressed
```
</TabItem>
</Tabs>
The API response will be
```json
{
"info": "file 'facial_speech_verbal_video_test.mp4' saved at 'files/facial_speech_verbal_video_test.mp4'"
}
```
Save the file location returned somewhere `files/facial_speech_verbal_video_test.mp4`
* Then you can use any API you want, insert the previous upload file url into `file_url` parameter. Then you can specify variables you want to get. For example, if you use the `/video/acoustic` API, you can specify the request body like
```json
[
"audio_intensity",
"pitch_frequency"
]
```
<Tabs
defaultValue="docs"
values={[
{label: 'API Docs', value: 'docs'},
{label: 'cUrl', value: 'curl'},
]}>
<TabItem value="docs">
<figure>
<img src="../docs/assets/opendbm_acoustic.png" width="700" alt="OpenDBM Test Acoustic API" />
<figcaption>ODBM Test Acoustic API</figcaption>
</figure>
</TabItem>
<TabItem value="curl">
```bash
curl 'http://127.0.0.1:8000/odbm/v1/video/acoustic?file_url=files%2Ffacial_speech_verbal_video_test.mp4&platform=local' \
-H 'Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiJhaWN1cmUiLCJleHAiOjE2NjI5MDkwMzd9.RQgtXhPCcgzctfuNg2GbByR9OMdr1vWTerl1iKPWqfk' \
-H 'Content-Type: application/json' \
-H 'accept: application/json' \
--data-raw $'[\n "audio_intensity",\n "pitch_frequency"\n]' \
--compressed
```
</TabItem>
</Tabs>
After you execute the API, you will get the zip file, with the csv results compressed inside that file
> You can check more details on rest_api usage in `rest_api/README.md`
## Technology Stack
<figure>
<img src="../docs/assets/odbm_tech_stack.png" width="500" alt="OpenDBM REST API Technology Stack" />
<figcaption>ODBM REST API Technology Stack</figcaption>
</figure>
The OpenDBM API will use FastAPI framework. The main reason is because the asynchronous behaviour of FastAPI will works perfectly to complex and high performance methods like data upload, data pre / processing and other possible long operations.
We also use Uvicorn is an ASGI web server implementation for Python. Until recently Python has lacked a minimal low-level server/application interface for async frameworks. The ASGI specification fills this gap, and means we're now able to start building a common set of tooling usable across all async frameworks. Uvicorn currently supports HTTP/1.1 and WebSockets.
## API Documentations
The Rest API documentations can be found when you run the the app. Change the relative path to /docs or /redoc
<figure>
<img src="../docs/assets/odbm_api_docs.png" width="500" alt="OpenDBM REST API Swagger API Docs" />
<figcaption>ODBM REST API Swagger API Docs</figcaption>
<img src="../docs/assets/odbm_redocs.png" width="500" alt="OpenDBM REST API Redocs" />
<figcaption>ODBM REST API Redocs</figcaption>
</figure>

8
docs/website/extras/opendbm-listserv-signup.md Normal file
View File

@@ -0,0 +1,8 @@
---
id: opendbm-listserv-signup
title: OpenDBM Listserv signup
---
[Use this link to sign up for the OpenDBM Listserv and stay up to date on release and events.](https://docs.google.com/forms/d/e/1FAIpQLScLrgyM08YIumzJw8dxmRaMQ4byBL3J4A90FpokDhX8-KUNZw/viewform)
From time to time, we will be making either small updates to OpenDBM functionalities (additional variables, bug fixes, etc.) or large version releases with major additions to features (e.g., we want a future update to OpenDBM to include analysis of wearable data). We will also be hosting virtual (and hopefully one day in-person) events for the OpenDBM user community that allow for training, introduction to new functionalities, presentations on prominent projects, etc. In either of these cases, we’d like to keep the user community up to date so that they don’t miss out on these updates or events, so we do encourage you to [sign up for the listserv.](https://docs.google.com/forms/d/e/1FAIpQLScLrgyM08YIumzJw8dxmRaMQ4byBL3J4A90FpokDhX8-KUNZw/viewform)

164
docs/website/extras/opendbm-python-api-add-new-variable.md Normal file
View File

@@ -0,0 +1,164 @@
---
id: opendbm-python-api-add-new-variable
title: OpenDBM API - Add New Variable
---
#### Prerequisites read
* [OpenDBM Python API](opendbm-python-api)
* [OpenDBM API - Unit Test](opendbm-python-api-unittest)
<figure>
<img src="../img/extras/api-add-var-5.png" width="70%" alt="OpenDBM Folder" />
<figcaption>Simple Architecure Diagram on How to Add New Module</figcaption>
</figure>
## Scenario
We want to add a pose detector in OpenDBM API. The model file that we will use to do the pose detector is 2 GB in size. Here is the steps on how to do it
### 1. Consider the category of pose detector
We have existing dbm group folders that consist of Facial, Movement, Speech, and Verbal Acoustics.
Compared to other categories, it will make sense if the pose detector is added to the Movement group. So with that in mind,
we will create the core module in the movement folder, to be precise, in **“dbm_lib/dbm_features/raw_features/movement”**.
For the module name, there is no convention on how to create the module name. The only important thing is it needs to be explicit.
In this case, we will name our module **pose_detection.py**
### 2. Create the core module
In general, this module is where we use the model to generate the final output as a pandas dataframe object.
Specifically, The final function of the pose_detection module needs to have two capabilities:
1. Keyword argument for saving the output.
This is essential because you need the module to be used with the Docker command approach. To develop this,
please use the save_output function from the util library so you don’t have to create the same function.
2. Should return the final output as pandas dataframe.
In code, the end of the final function should look like this.
<figure>
<img src="../img/extras/api-add-var-1.png" width="100%" alt="OpenDBM Folder" />
<figcaption>End Code of eye_blink Module in Movement dbm group</figcaption>
</figure>
### 3. Store the Model File
Now store your model file in the pkg/local folder, depending on the file size (see pkg folder explanation here). You
also need to create a function where openDBM API can download the model file if the user doesn’t have it yet.
The function must be made in the **util.py** module in the api_lib folder.
### 4. Add pose_detection to Python API
Below are the steps to add your newly created pose_detection module to Python API that lives inside api_lib folder.
#### a. Create the private module.
Based on our scenario, we should create “_pose_detection.py” in the movement folder inside api_lib
#### b. Create a class inside the private module
If we look at all the private modules, the code has the same pattern as others. So all we need to do is to modify it a little bit.
Below is the code inside “_eye_blink.py” inside the movement folder.
<figure>
<img src="../img/extras/api-add-var-2.png" width="100%" alt="_eye_blink.py" />
<figcaption>_eye_blink.py</figcaption>
</figure>
As a starter, we copy-paste all codes from “_eye_blink.py” to “_pose_detection.py”. Then in;
* Line 1: change DLIB_SHAPE_MODEL to POSE_DETECTION_MODEL. This object is a path to where the Pose detector model lives.
* Line 2: change the import module from eye_blink to pose_detection, and also change run_eye_blink to your final function name (ex: run_pose_detection)
* Line 5: Change the class name from EyeBlink to PoseDetection
* Line 8: create a list that contains the name of the numerical field of your processed dataframe.
* Line 11: change the _fit_transform method to return the result of your function, in this case, run_pose_detection(args1,args2, …)
The final codes in **_pose_detection.py** should be look like this, for example;
```python
from opendbm.api_lib.model import POSE_DETECTION_MODEL, VideoModel
from opendbm.dbm_lib.dbm_features.raw_features.movement.pose_detection import run_pose_detection
class PoseDetection(VideoModel):
def __init__(self):
super().__init__()
self._params = ["mov_pose_head", "mov_pose_body", "mov_pose_hand", "mov_pose_foot"]
def _fit_transform(self, path):
return run_pose_detection(path, ".", self.r_config, POSE_DETECTION_MODEL, save=False)
```
#### c. Export the newly created module to api.py
Now we go and take a look at codes in module api.py. After that, all we have to do is add pose_detection with the following steps:
#### Import the private pose module
Following the pattern, we just need to add **from ._pose_detection import PoseDetection**
```python
from ._eye_blink import EyeBlink
from ._eye_gaze import EyeGaze
from ._facial_tremor import FacialTremor
from ._head_movement import HeadMovement
from ._vocal_tremor import VocalTremor
from ._pose_detection import PoseDetection # <-- new code added for pose_detection
```
#### Initiate the model class
Following the pattern, create a new variable in a new line **self._pose_detection = PoseDetection()**
```python
class Movement(VideoModel):
def __init__(self):
super().__init__()
self._eye_blink = EyeBlink()
self._eye_gaze = EyeGaze()
self._facial_tremor = FacialTremor()
self._head_movement = HeadMovement()
self._vocal_tremor = VocalTremor()
self._pose_detection = PoseDetection() # <-- new code added for pose_detection
```
#### Add the new pose variable to models dictionary.
```python
self._models = OrderedDict(
{
"eye_blink": self._eye_blink,
"eye_gaze": self._eye_gaze,
"facial_tremor": self._facial_tremor,
"head_movement": self._head_movement,
"vocal_tremor": self._vocal_tremor,
"pose_detection": self._pose_detection # <-- new code added for pose detection
}
)
```
#### Create a new method to get the result of pose_detection.
By following the function pattern in the code below, our function name will be **get_pose_detection**, which will return **self._pose_detection**
```python
def get_vocal_tremor(self):
"""
Get the model object of Vocal Tremor
Returns:
self: object
Model Object
"""
return self._vocal_tremor
### Below is the new code to get the result of pose_detection
def get_pose_detection(self):
"""
Get the model object of Pose Detection
Returns:
self: object
Model Object
"""
return self.pose_detection
```
That is how to add a new component to OpenDBM API. The only leftover is,
### 5. Add pose_detection to unit test (pytest).
Considering you have already read the Unit test design, you need to create a function to test your newly created API.
In our case, in “test_api_movement.py,” we will create a test function named “test_get_pose_detection,” and you should configure your unit testing there.

34
docs/website/extras/opendbm-python-api-unittest.md Normal file
View File

@@ -0,0 +1,34 @@
---
id: opendbm-python-api-unittest
title: OpenDBM API - Unit Test
---
#### Prerequisites read
* [OpenDBM Python API](opendbm-python-api)
The framework we use to do unit testing is pytest. We use pytest because we want our test design to be very similar to our API design and make the unit test easier to understand. Below is the general architecture of the tests folder.
<figure>
<img src="../img/extras/api-test-docs-1.png" width="100%" alt="Unit Tests Folder" />
<figcaption>Unit Tests Folder</figcaption>
</figure>
The picture above contains the test folder containing the test_data folder that will store many test files.
All the files here will be used as test data during our unit testing.
Next to the test_data folder is conftest.py, the pytest configuration that will be applied to all the test scripts. This file's purpose is to initiate all the API, so we don't have to import it into our test script.
Lastly, we have dbm group folders (facial, movement, speech, verbal acoustics).
Each of these folders consistently has the same structure, which consists of the following:
### 1. Test Script
All test scripts file that has the prefix “test_” is where we create all our unit tests, respective to the folder.
(ex: test script inside the facial folder should only have unit test related to the facial API)
### 2. Configuration Test
This file named conftest.py in each folder will configure how many test data files you want to add to the test script.
If you're going to add another test data, you should create a new function inside this file to feed it into the test script.

101
docs/website/extras/opendbm-python-api.md Normal file
View File

@@ -0,0 +1,101 @@
---
id: opendbm-python-api
title: OpenDBM Python API
---
When we use OpenDBM python API, all the source codes come from the opendbm folder in our GitHub repo. opendbm folder is the main
package we use to process input data. opendbm has four folders that consist of api_lib, dbm_lib, pkg, and resources.
To make it clear in the beginning, you will know that our structure in our repo reflects what is already documented in <a href="../docs/biomaker-variables">Variables</a> Section.
<figure>
<img src="../img/extras/api-docs-1.png" width="100%" alt="OpenDBM Folder" />
<figcaption>OpenDBM Folder</figcaption>
</figure>
### 1. dbm_lib
<figure>
<img src="../img/extras/api-docs-2.png" width="100%" alt="dbm_lib Folder" />
<figcaption>dbm_lib folder @ dbm_lib/dbm_features/raw_features</figcaption>
</figure>
When we mention dbm_lib for python API documentation, we refer to the folder “dbm_lib/dbm_features/raw_features.”
dbm_lib is where all core modules are implemented. All the processing components must be stored here because this
folder is where we can use the components either with the Docker command or Python API
Each children's folder (audio, movement, nlp, and video) represents the dbm group. In every children folder,
there will be modules where we perform the processing/inferring of our input data.
For example, below are the modules that live inside the movement folder.
<figure>
<img src="../img/extras/api-docs-3.png" width="100%" alt="movement Folder in dbm_lib" />
<figcaption>movement Folder in dbm_lib</figcaption>
</figure>
The goal of each module here is to perform one task/processing. For instance, facial_tremor.py
will have a functionality to detect the facial tremor of the input data. If we want to add another functionality
in openDBM, for example, motion detection, we should create a new module in this folder (assumed motion detection
is part of movement dbm_group).
The design we explained for the movement package also applied to other dbm groups such as video, nlp, and audio.
Lastly, in the same level of the dbm group folder, there is the util folder, which contains all utility functions used
by all modules that need it. If we want to create/modify the modules in dbm_lib, please check all the utility
functions first so we don't need to reinvent the wheel.
### 2. api_lib
<figure>
<img src="../img/extras/api-docs-4.png" width="100%" alt="api_lib Folder" />
<figcaption>api_lib Folder</figcaption>
</figure>
api_lib is a supporting package created so that we can use openDBM core modules with the python API call.
Looking in the api_lib folder, we can see four folders (facial_activity, movement, speech, verbal_acoustics),
which represent the dbm group, the same as we saw in the dmb_lib folder. In the same level of the dbm group folder,
there are also two modules that have functionality as follows:
#### a. model.py
This module has the functionality to create the base model object that can be used after we execute the fit method
in our Python API. This module also has a class to perform a processing/fit method in a dbm_group level.
#### b. util.py
This module is where all the utility functions are stored, related to the api_lib folder.
<figure>
<img src="../img/extras/api-docs-5.png" width="100%" alt="movement Folder in api_lib" />
<figcaption>movement Folder in api_lib</figcaption>
</figure>
All the dbm group folders inside api\_lib consistently have the same design as other folders.
If we look in the movement folder, we can see all the private modules (with the prefix “\_”) that have
the same name/structure as the one in the dbm_lib folder.
All these private modules will execute the same as their respective modules in the dbm_lib folder. For example, module “_vocal_tremor.py” will execute processing “vocal_tremor.py”. The difference is that in these private modules, we assign the function to the class that will be used for our python API.
The “api.py” module is a module where we implement the python API call tailored to each of the dbm_group.
### 3. pkg
is where the model files are stored. In most of our processing/inferring, we get the weight from this pkg folder.
<b>Note for the developer</b>; if you want to make the user download a new model file, you can direct the download path to the pkg folder. But if the model file is big (ex: 2GB), please consider directing the download path to their local path. The reason is users might try to use OpenDBM in many virtual environments. Their memory storage will be exhausted if we store large model files in the pkg folder in many virtual environments.
### 4. resources
This folder is created initially as a configuration to rename, clean, and structure OpenFace output. To make it consistent, if we need to add other configurations for processing a new model, please create it in this folder.

8
docs/website/extras/publications-and-referencing.md Normal file
View File

@@ -0,0 +1,8 @@
---
id: publications-and-referencing
title: Publications and Referencing
---
As the user community grows, we hope and are anticipating scientific publications whose methods depend partially or entirely on methods compiled into OpenDBM. For the authors of those publications, we request that they reference OpenDBM in their work. We are (as of December of 2020) preparing a manuscript that can be referenced. Meanwhile, we ask that authors at the very least reference the Github repository when mentioning OpenDBM.
Additionally, we are hoping to maintain an up to date list of publications that have used methods from OpenDBM when reporting their findings. This live list of publications is linked to from many different places around the Github repository and this documentation, but [here is another link to it](https://docs.google.com/spreadsheets/u/2/d/1pRBWCCFMbEgZNQzm2Litm3RUQ6glwUwswrqGDePMvh0/edit#gid=0). If you are a user of OpenDBM and have used its methods in work that has either been presented at a conference or submitted to a journal, we ask that you please submit a citation of the work to us [using this form](https://forms.gle/Hb6bDL1GJvG1ByUX7) so that we may append the list of publications with your work.