Skip to content

add preservation of comment lines#20

Open
wififreedom wants to merge 9 commits intoopenwrt:masterfrom
wififreedom:preserve_comments
Open

add preservation of comment lines#20
wififreedom wants to merge 9 commits intoopenwrt:masterfrom
wififreedom:preserve_comments

Conversation

@wififreedom
Copy link
Copy Markdown

@wififreedom wififreedom commented Mar 31, 2026

This pull request implements preservation of comments in configuration files.

Comment lines before an entry, and the comment at the end of an entry are considered to be associated with that entry.

Example:

# this comment line is associated with section2

  # this comment line is associated with section2
config type section2 # this comment is associated with section 2
# this comment line is associated with opt
    
        # this comment line is associated with opt
        option opt 'val' # this comment is associated with opt
    
      # this comment line is not associated with any entry

Wih option -D, uci works as before and strips comments. The following commands have enhanced behaviour:

  • uci import preserves comments of unchanged configuration entries, and imports entries with their associated comments
  • uci export exports entries with their associated comments
  • uci 'commit preserves comments of unchanged configuration entries when committing changes with uci commands 'add', 'delete', 'set', 'add_list', and/or 'del_list'.
  • uci get '<config>.<section>' '#' shows the section comment
  • uci get '<config>.<section>.<option> '#' shows the option comment
  • uci set '<config>.<section>=<value>' '<comment>' sets the section value and comment
  • uci set '<config>.<section>.option=<value>' '<comment>' sets the option value and comment
  • uci add_list '<config>.<section>.option=<value> '<comment>' adds a list entry with value and comment
  • uci changes also shows comments
  • uci show does not show comments
  • uci batch supports the above commands

Features:

  • Comments are auto-indented if preserved, set or added.
  • The new delta file format is both backward compatible and forward compatible. Older uci versions interpret the addition in the new format as comment and ignores it.
  • Comments at the end of the file are not preserved because they do not belong to an entry.

I've added lots of shunit2 tests to verify the implementation.

An example of a result: the above example after and option 'opt2' has been added:


# this comment line is associated with section2
# this comment line is associated with section2
config type section2 # this comment is associated with section 2
        # this comment line is associated with opt
        # this comment line is associated with opt
        option opt 'val' # this comment is associated with opt
        # comment line for new option 'opt2'
        option opt2 'val2'

@GeorgeSapkin
Copy link
Copy Markdown
Member

You'll need to look into the submission guidelines regarding commit formatting, etc.

@wififreedom
Copy link
Copy Markdown
Author

wififreedom commented Apr 1, 2026

I will rebase with a proper commit message once I'm satisfied.

@wififreedom
Copy link
Copy Markdown
Author

wififreedom commented Apr 2, 2026

The shunit2 test is broken, does not fail if one of the subtests fails, because exit code 1 is not propagated.

In tests/shunit2/tests.sh:

/bin/sh ${FULL_SUITE}

rm -rf ${TESTS_DIR}

should be:

/bin/sh ${FULL_SUITE}
EXITCODE=$?

rm -rf ${TESTS_DIR}

exit ${EXITCODE}

See #21

@wififreedom wififreedom marked this pull request as ready for review April 2, 2026 14:16
@wififreedom wififreedom force-pushed the preserve_comments branch 3 times, most recently from 92f0747 to bf0a598 Compare April 3, 2026 11:21
@wififreedom wififreedom marked this pull request as draft April 4, 2026 09:06
shunit2 test with failing subtest incorrectly produced:

      Start 2: shunit2
  2/2 Test openwrt#2: shunit2 ..........................   Passed   52.70 sec

  100% tests passed, 0 tests failed out of 2

now correctly produces:

      Start 2: shunit2
  2/2 Test openwrt#2: shunit2 ..........................***Failed   53.68 sec
  #
  # Performing tests
  #
  ...
  test_get_option
  ASSERT:expected:<badval> but was:<val>
  test_get_option_multiline
  ...

  #
  # Test report
  #
  tests passed:   111  99%
  tests failed:     1   1%
  tests skipped:    0   0%
  tests total:    112 100%

  50% tests passed, 1 tests failed out of 2

  Total Test time (real) =  59.67 sec

  The following tests FAILED:
	2 - shunit2 (Failed)
  Errors while running CTest
  make: *** [Makefile:74: test] Error 8
  make: Leaving directory '/home/user/repos/uci/build'

Signed-off-by: Bastiaan Stougie <wififreedom2026@protonmail.com>
Comment lines before an entry, and the comment at the end of an entry
are considered to be associated with that entry.

Example:

  # this comment line is associated with section2

    # this comment line is associated with section2
  config type section2 # this comment is associated with section 2
  # this comment line is associated with opt

	  # this comment line is associated with opt
	  option opt 'val' # this comment is associated with opt

  # this comment line is not associated with any entry

Added code to optionally, for each entry in the configuration file:
- parse the associated comments, and store them in buffer 'commentbuf'
  in the parse context.
- empty commentbuf after the associated entry has been processed.

Signed-off-by: Bastiaan Stougie <wififreedom2026@protonmail.com>
@wififreedom wififreedom force-pushed the preserve_comments branch 6 times, most recently from 56181fc to 0cb0626 Compare April 6, 2026 22:06
@wififreedom
Copy link
Copy Markdown
Author

This is ready for review.

@wififreedom wififreedom marked this pull request as ready for review April 6, 2026 22:33
@wififreedom
Copy link
Copy Markdown
Author

wififreedom commented Apr 8, 2026

I've just locally built, for 25.12.2, x86_64 target:

  • package/system/uci with the changes of this pull request
  • package/utils/ucode
  • package/network/utils/iwinfo
  • package/system/rpcd with a change in uci.c: cursor->flags |= UCI_FLAG_COMMENTS;
  • package/feeds/luci/rpcd-mod-luci

This produced the following apk files that are all needed:

libiwinfo-data-2026.01.14~f5dd57a8-r1.apk
libiwinfo20230701-2026.01.14~f5dd57a8-r1.apk
libuci20260406-2026.04.06~0cb06268-r1.apk
rpcd-2026.04.08~d25812b7-r1.apk
rpcd-mod-file-2026.04.08~d25812b7-r1.apk
rpcd-mod-iwinfo-2026.04.08~d25812b7-r1.apk
rpcd-mod-luci-20240305-r1.apk
rpcd-mod-rpcsys-2026.04.08~d25812b7-r1.apk
rpcd-mod-ucode-2026.04.08~d25812b7-r1.apk
uci-2026.04.06~0cb06268-r1.apk
ucode-mod-uci-2026.01.16~85922056-r1.apk

Installed these on my system, restarted service rpcd, added comments to /etc/config/network, made changes via Luci -> Network -> Interfaces, Save & Apply ... and the comments were preserved.

@wififreedom
Copy link
Copy Markdown
Author

I would like to enable comment preservation by default, but I'm cautious, for the following reasons:

  • uci and libuci consume less memory by discarding comments (not sure if relevant, depends on how many comments there are)
  • tests (for other packages) that expect comments to disappear after using uci no longer work

But I'm open to enabling comment preservation by default, and optionally adding flag UCI_FLAG_STRIP_COMMENTS instead of UCI_FLAG_COMMENTS, and a command line option to strip comments instead of option '-k'.

@wififreedom
Copy link
Copy Markdown
Author

I've decided to enable preservation of comments by default. The use cases against disabling it by default (memory usage and some tests that may need to be adapted) are very unlikely a problem. It can still be disabled.

With option '-k':
- uci command 'import' preserves comments of unchanged configuration
  entries, but does not import comments
- uci command 'export' exports entries with their associated comments
- uci command 'commit' preserves comments of unchanged configuration
  entries when commiting changes with uci commands 'add', 'delete',
  'set', 'add_list', and/or 'del_list'.
- preserved comment lines are auto-indented

Limitation:
- comments at the end of the file are not preserved, because they are
  not associated with an entry

Signed-off-by: Bastiaan Stougie <wififreedom2026@protonmail.com>
Added shunit2 tests for option '-k': keep comments for unchanged
entries.

Signed-off-by: Bastiaan Stougie <wififreedom2026@protonmail.com>
With option '-k':
- uci command 'import' now also imports comments associated with
  imported entries
- imported comments are auto-indented

Limitation:
- comment lines at the end of the file are not imported

Signed-off-by: Bastiaan Stougie <wififreedom2026@protonmail.com>
Test that 'import' with option '-k' also imports comments associated
with imported entries.

Signed-off-by: Bastiaan Stougie <wififreedom2026@protonmail.com>
- `uci -k get '<config>.<section>' '#'` shows the section comment
- `uci -k get '<config>.<section>.<option> '#'` shows the option comment
- `uci -k set '<config>.<section>=<value>' '#<comment>'` sets the section
   value and comment
- `uci -k set '<config>.<section>.option=<value>' '#<comment>'` sets the
   option value and comment
- `uci -k add_list '<config>.<section>.option=<value> '#<comment>'` adds
   a list entry with value and comment
- `uci -k changes` also shows comments
- `uci -k show` does not show comments
- `uci -k batch` supports the above commands
- comments are auto-indented if set or added

Signed-off-by: Bastiaan Stougie <wififreedom2026@protonmail.com>
- `uci -k get '<config>.<section>' '#'`
- `uci -k get '<config>.<section>.<option> '#'`
- `uci -k set '<config>.<section>=<value>' '#<comment>'`
- `uci -k set '<config>.<section>.option=<value>' '#<comment>'`
- `uci -k add_list '<config>.<section>.option=<value> '#<comment>'`
- `uci -k changes`
- `uci -k show`
- `uci -k batch`

Signed-off-by: Bastiaan Stougie <wififreedom2026@protonmail.com>
Comments are now preserved by default by uci and libuci. There are
a lot of programs that would otherwise need to be modified to
explicitly set a flag to preserve comments.
- replaced UCI_FLAG_COMMENTS by UCI_FLAG_NO_COMMENTS (to be able to
  explicitly request the old behavior, if you really want to strip
  comments for some reason).
- replaced option -k (keep comments) with option -D (don't keep
  comments)
- also fixed a pre-existing memory leak in cli.c:main() on invalid
  option.

Signed-off-by: Bastiaan Stougie <wififreedom2026@protonmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants