Monday, 25 May 2026

Filled under:

 1. Does Liquibase run automatically during every application startup?

2. Is Liquibase enabled on all application nodes?

3. Can more than one app node attempt Liquibase migration at the same time?

4. Was deployment/startup happening when the host was shut down?

5. Is the application connecting through HAProxy/PGPool/service name or directly to a Patroni node?

6. After failover, does the app retry Liquibase against the new primary?

7. Do they have an approved release-locks procedure?



1. Stop application services

2. Confirm no Liquibase/deployment process is running

3. Confirm active DB sessions are drained

4. Stop Patroni/PostgreSQL gracefully on target node

5. Perform host maintenance

6. Start Patroni/PostgreSQL

7. Confirm cluster role and health

8. Start application services

9. Validate Liquibase lock and application startup



databasechangelog      -- history of successfully executed Liquibase changesets

databasechangeloglock  -- current lock status used to control Liquibase execution



atabasechangeloglock.locked = false

Then there may be no Liquibase lock issue currently.


In that case, application team may be assuming incorrectly. The startup failure may be somewhere else, such as:

DB connection issue
Patroni primary/replica routing issue
Application connecting to replica
Schema mismatch
Permission issue
Liquibase disabled/not running
Application startup dependency failure



The databasechangelog table showing only 2025 dates means no recent Liquibase changeset has been successfully applied. However, that table is only the execution history. The current lock status is controlled by databasechangeloglock.

Please share the output of select * from databasechangeloglock;, especially locked, lockgranted, and lockedby. If locked=true with a recent timestamp, then Liquibase likely acquired the lock during app startup/deployment and got interrupted before releasing it. If locked=false, then this may not be a Liquibase lock issue and we should check app connectivity, Patroni routing, and whether the app is connecting to primary or replica.







Hi Team, sharing the initial understanding on the reported databasechangeloglock concern.

Application team observed that after an abrupt host shutdown/crash scenario, the application may face issues related to the Liquibase lock table. However, current DB-side validation shows databasechangelog has only older 2025 entries, meaning no recent Liquibase changeset appears to have completed. Also, databasechangeloglock currently shows locked = f, so there is no active stuck Liquibase lock at database level at this point.

Based on this, we should not conclude yet that this is a PostgreSQL/Patroni defect. The more likely area to validate is whether Liquibase/application startup was interrupted during host shutdown, whether Liquibase runs during app startup, whether multiple app nodes can trigger it, and whether the application was connected to the correct Patroni primary after recovery/failover.

From DB side, we can validate PostgreSQL/Patroni logs, crash recovery/failover events, current primary role, and connection routing. Application team should share exact startup/deployment error logs and confirm Liquibase execution flow.

We can still raise a passive/vendor case for guidance on PostgreSQL/Patroni behavior during abrupt shutdown, but the case should be positioned as validation support rather than concluding PostgreSQL is updating the Liquibase lock flag.





. Also, databasechangeloglock currently shows locked = f, so there is no active stuck Liquibase lock at database level at this point

So at present, there is no active database-side Liquibase lock issue.


Based on the current evidence, the issue cannot yet be concluded as a PostgreSQL or Patroni defect. The databasechangeloglock table is managed by Liquibase/application deployment logic, and a stuck lock generally occurs when Liquibase acquires the lock and the application/Liquibase process is interrupted before it can release it. Therefore, the current symptoms need to be validated from both the application startup/deployment flow and the PostgreSQL/Patroni recovery/failover perspective.


-


Hi Team, sharing the initial understanding on the reported databasechangeloglock concern.

Application team observed that after an abrupt host shutdown/crash scenario,

the application may face issues related to the Liquibase lock table.

Current understanding is

If Liquibase/application startup is running during an abrupt shutdown, Liquibase may get interrupted while holding the lock. In such a case, the lock can remain stuck until released safely.

So at present, there is no active database-side Liquibase lock issue.

However, I tried checking a few things right now and current DB-side validation shows databasechangelog has only older 2025 entries,

meaning no recent Liquibase changeset appears to have completed



The application team should validate:

Based on this, we should not conclude yet that this is a PostgreSQL/Patroni defect. The more likely area to validate is whether Liquibase/application startup was interrupted during host shutdown, whether Liquibase runs during app startup, whether multiple app nodes can trigger it, and whether the application was connected to the correct Patroni primary after recovery/failover.


From DB side, we can validate PostgreSQL/Patroni logs, crash recovery/failover events, current primary role, and connection routing. Application team should share exact startup/deployment error logs and confirm Liquibase execution flow.



1. Whether Liquibase runs automatically during application startup.
2. Whether Liquibase runs from multiple application nodes.
3. Whether the application was starting or deploying when the host shutdown happened.
4. Whether any Liquibase process was interrupted.
5. Whether application logs show "Waiting for changelog lock" or "Could not acquire changelog lock".
6. Whether the application was connected to the Patroni primary or accidentally routed to a replica.
7. Whether the application has a safe Liquibase release-locks procedure.


At this stage, there is no active database-side lock to clear, and no recent Liquibase changeset execution is visible from `databasechangelog`. The next step is to validate the application/Liquibase logs around the incident timestamp to confirm whether Liquibase was waiting for a changelog lock, failed before applying changes, or was interrupted during startup. From the database side, we should validate Patroni/PostgreSQL logs to confirm crash recovery, switchover timing, current primary role, and application connection routing. A PostgreSQL vendor case may be raised only as a passive validation case for Patroni/PostgreSQL behavior during abrupt shutdown/switchover. However, based on current evidence, this should first be investigated as an application/Liquibase startup and recovery flow issue rather than a confirmed PostgreSQL defect.


Current DB state does not show an active Liquibase lock issue. The old 2025 entries in `databasechangelog` mean no recent Liquibase changeset was successfully applied. The current `locked = f` means the lock is clear now. But because switchover happened today, we cannot confirm the lock state before switchover. Therefore, the RCA should focus on timeline correlation and application/Liquibase logs before raising this as a PostgreSQL issue.


SELECT pid, usename, application_name, client_addr, state, query_start, wait_event_type, wait_event, query FROM pg_stat_activity WHERE query ILIKE '%databasechangelog%' OR query ILIKE '%liquibase%' OR application_name ILIKE '%liquibase%';

0 comments:

Post a Comment