The “Keep settings” checkbox is a more advanced feature. It performs OpenWrt backup before upgrade and restores it after upgrade.
If you do not precisely understand the button's use cases, uncheck “Keep Settings” every time you flash a new OpenWrt sysupgrade to your device, to not preserve settings.
Despite the warnings on this page, keeping settings in sysupgrade works for most of the times.
See also: Backup and restore