Configuration File¶

Create¶

To create a new configuration file then run the script with a create configuration file option, ‘-c’:

thegmu_nextcloud_git_sync.py -c

# On host named 'fuzzyfinehat'.
% python3 thegmu_nextcloud_tools/thegmu_nextcloud_git_sync.py -c
% thegmu_nextcloud_git_sync.fuzzyfinehat.yaml file created.

Sample¶

The created configuration file name includes the host name. If you have more than one configuration file you’ll to change the file name. Sample:

% python3 thegmu_nextcloud_tools/thegmu_nextcloud_git_sync.py -c
% thegmu_nextcloud_git_sync.fuzzyfinehat.yaml file created.
% cat thegmu_nextcloud_git_sync.fuzzyfinehat.yaml
name: 'thegmu-nextcloud-git-sync'
davfs:
  mount:
    source_url: 'https://www.yourdomain.com/oc/remote.php/dav/files/youraccount/
    target_dir: '/media/davfs/yournextcloudaccount'
    max_active_seconds: 30
nextcloud:
  account:
    linux: 'www-data'
    nextcloud: 'yournextcloudaccount'
  directory:
    exclude:
      - 'Transfer'
    include: null
    root: '/path/to/nextcloud/data/yournextcloudaccount/files'
git:
  account:
    linux: 'yourlinuxaccount'
  directory:
    exclude:
      - '.git'
      - '__pycache__'
      - 'Transfer'
    include: None
      root: '/home/yourlinuxaccount/path/to/git/repo'
  pull_cmd: 'sudo su - %s -c "cd %s && git pull"'
report:
  file:
    process: '/path/to/your/log/dir/thegmu_nextcloud_git_sync.processed.csv'
    error: '/path/to/your/log/dir/thegmu_nextcloud_git_sync.errors.csv'
  error:
    columns:
    - 'File'
    - 'Time'
    - 'Cause'
  column:
    timestamp: 'Time'
run:
  production_host: 'nextcloudhost'
  lockpath: '/tmp/thegmu_nextcloud_git_sync.py.lock'
test:
  account:
    linux: 'yourlinuxtestaccount'
  directory:
    run: '/home/yourlinuxtestaccount/path/to/test/git/repo'
    temp_test: "/home/yourlinuxtestaccount/path/to/mock/nextcloud/data"
  file:
    delete: '/home/yourlinuxtestaccount/path/to/mock/nextcloud/data/youraccount/files/deleteme.txt'
    temp_test_nextcloud_data: "/home/yourlinuxtestaccount/path/to/test/git/repo/archive/repo.tgz"
    zero_file: '/home/yourlinuxtestaccount/path/to/mock/nextcloud/data/youraccount/files/zero_file.txt'
  mount:
    target_dir: '/media/davfs/youratestccount'

DavFS¶

Install DavFS if not already installed. On Ubuntu:
```
sudo apt install davfs2
```
Create the mount point. On Ubuntu create a davfs directory under /media and then the nextCloud account:
```
sudo mkdir -p /media/davfs/yournextcloudaccount
```
nextCloud account DavFS URL and password. The credentials for the nextCloud account are kept in the DavFS secrets file, /etc/davfs/secrets. To mount the DavFS then three pieces of information are required in the secrets file: account name, account password and URL. #. Log into the nextCloud account and in the lower righthand corner of the main click on the ‘Settings’ wheel icon. A dialog will open that includes the DavFS URL:
```
https://www.thegmu.com/oc/remote.php/dav/files/gmucorp/
```
Update DavFS credentials in the secrets file, /etc/davfs/secrets. Follow the instructions provided at the top of the file for adding the mount point entry.

Test DavFS mount:

sudo mount /media/davfs/yournextcloudaccount

Update DavFS entries in the configuration file. Edit the configuration files and update the davfs section with source_url, and target_dir. Also add the nextCloud account to the nextcloud section.

Note

The davfs section, entry of max_active_seconds of 30 seconds is recommended when running synchronization every minute. You may increase the max_active_seconds to reflect longer synchronization times. You should test your DavFS implementation using a copy set of files that takes longer than the max_active_seconds and monitor the nextCloud performance. Our tests using rsync with gigabytes of files would cause the DavFS mount to hang.

nextCloud¶

Accounts: There are two account entries for the nextcloud section, the nextCloud user account and the Linux account. The Linux account represents the owner of the disks files. On Ubuntu/Apache the user is www-data with a default nextCloud installation.
Directories: Update the root directory and the include, exclude directories given the answers to your policy questions and your nextCloud installation. Just realize that the root path is to the nextCloud specific account files and not the nextCloud server data root.

GIT¶

Accounts: There is one account entry for the git section, Linux account that owns the git repo mirror. The Linux account represents the owner of the disks files.
Directories: Update the root directory and the include, exclude directories given the answers to your policy questions and your GIT repo requirements.
pull_cmd: Updates to GIT changes are done using GIT pull. The template configured command should work unless the version such as branch need to be added.

Note

The use of sudo to specify the Linux account ensures the root user account will work as well.

Report¶

Files: If you only have one configuration file configured then the default file names will suffice, just update the path to reflect your policy. If multiple configuration files are used then each configuration file needs a distinct set of files by either specifying distinct directories or distinct file names per your policy.
Error: no configuration required.
Column: no configuratoin required.

Run¶

production_host: This must be the current host of the nextCloud host. If the production_host does not match the current host then the test configuration is used.

Warning

This must be set to the nextCloud host or a test run is executed using the values in the test section of the configuration file.

lockpath: No change is required.

Test¶

No configuration required.

Test Configuration¶

Test your configuration by runing the script passing the configuration file:

sudo thegmu_nextcloud_git_sync.py thegmu_nextcloud_git_sync.yourhost.yaml

Next up is creating the cron scripts for synchronization every minute and optionally to send a daily email report of files processed for a day.