When you are creating the db instance you will find a checkbox that will enable automatic minor version upgrades like the following image.
We can find out if the option is enabled by running the following command:
aws rds describe-db-instances --db-instance-identifier <your database name> --region <your region> --query 'DBInstances[*].AutoMinorVersionUpgrade'
Or by going to the RDS console, clicking on the db instance and select the Maintenance & backups tab.
In this case, you have not enabled Auto minor version upgrade in the first create database.
Now, you can enable by click Modify button, then scroll down and select Enabled Auto minor version upgrade
A PostgreSQL DB instance is automatically upgraded during your maintenance window if the following criteria are met: