Can I force delete an AWS CloudFormation stack that is In Progress of Rollback

Amazon Cloudformation

Amazon Cloudformation Problem Overview


An AWS CloudFormation rollback (e.g., UPDATE_ROLLBACK_IN_PROGRESS) has been in progress forever, like over an hour and a half. I want to delete the stack altogether or force stop any activity. Is this possible?

Thanks!

Amazon Cloudformation Solutions


Solution 1 - Amazon Cloudformation

Another common cause of blocked stack updates/rollbacks is errors in ECS::Service resource updates: it doesn't look like that is currently detected (in some cases?). Cloudformation is waiting for the service event for the service reaching a steady state, so simply updating the service to something that works (e.g. desired tasks to 0) will unblock it. Try to get the state back to what Cloudformation expects before sending more updates to avoid problems, though.

Solution 2 - Amazon Cloudformation

I guess your stack resources are changed or deleted by outside. You can find official guide as below.

>Manually sync resources so that they match the original stack's template, and then continue rolling back the update. For example, if you manually deleted a resource that AWS CloudFormation is attempting to roll back to, you must manually create that resource with the same name and properties it had in the original stack.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html#troubleshooting-errors-update-rollback-failed

or (as @talentedmrjones said)

>To fix the stack, contact AWS customer support.

https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html#troubleshooting-errors-nested-stacks-are-stuck

In my case, I can stop same situation via re-creating deleted resource.

Solution 3 - Amazon Cloudformation

In my case it is an EC2 security group that cannot be deleted because it is referenced from another EC2 security group.

Solution 4 - Amazon Cloudformation

When dealing with a custom resource it is possible to construct a mocked up version of the return url.

The easiest way to do this is to grab the url which was used during the create. If you can get your hands on it, replace the section after the last %2F with the "Client Request Token" which you can get from your event log for the cloudformation.

If not, then here's the format of the url you'll have to construct.

https://{region}.console.aws.amazon.com/cloudformation/home?region={region}#/stacks?filter=active&tab=events&stackId={stack arn}%2F{stack name}%2F{client request token}

Run that url as a get and it will cause the resource to fail rollback or delete.

Solution 5 - Amazon Cloudformation

Sometimes this will occur if your user role is missing permissions to delete roles. This can be tested by trying to manually delete roles or users that have been created by the CloudFormation stack.

Solution 6 - Amazon Cloudformation

I had something like this happen once, and the stack seemed stuck forever in UPDATE_ROLLBACK_IN_PROGRESS status. I'd recommend submitting a ticket to AWS support. That was the only way I was able to resolve it.

Solution 7 - Amazon Cloudformation

You can try to delete the resources and then the update rollback will complete successfully.

Solution 8 - Amazon Cloudformation

Was able to delete mine by manually deleting everything via AWS dashboard. I ended up having a couple dangling roles that just needed deletion.

Solution 9 - Amazon Cloudformation

I meet the same problem. The console told me some resource depends on another, so can't be deleted. Under that state, rollback in unavailable. I just delete the whole VPC and the resources in that VPC. Because cloudformation will retry to delete resource every 10-20 min. So when it retry, it will find the resource have already been deleted, and it just skip the deletion and everything is smooth after that.

Solution 10 - Amazon Cloudformation

Yes, use this command to delete stacks stuck in 'DELETE_IN_PROGRESS' state. You can easily run this in AWS CloudShell also.

Go to Lambda Function->Monitor->CloudWatch Logs. Look for Log where "RequestType" is "Delete" and Copy the necessary fields to below command

curl -H 'Content-Type: ''' -X PUT -d '{"Status": "SUCCESS","PhysicalResourceId": "Add your physical resource ID",	"StackId": "Add your StackId","RequestId": "Add your RequestID","LogicalResourceId": "LambdaFunction"}' 'Add your ResponseURL Here'

Example:

curl -H 'Content-Type: ''' -X PUT -d '{"Status": "SUCCESS","PhysicalResourceId": "cutomRes-LambdaFunction-1NC1ORF",	"StackId": "arn:aws:cloudformation:us-east-1:3343:stack/cutomRes/f52a-11eb-b5df-0a5c2cc1","RequestId": "d70931a2-364b-413e-a2","LogicalResourceId": "LambdaFunction"}' 'https://cloudformation-custom-resource-response-useast1.s3.amazonaws.com/arn%3Aaws%/cutomRes/f5466f6Expires=7200&X-Amz-Credential=AKIA6L7Q4OWT3GW5BT7K%2F20210330%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Signature=1db1f83f'

Do Note that, example contains URL that mmaybe modified to not work for security purposes. It is for demonstration purposes only.

Solution 11 - Amazon Cloudformation

You will need to investigate why exactly the rollback is taking so long (e.g., if it's due to a missing resource modified outside of the CloudFormation stack, or a Custom Resource that failed to return the expected signals).

Solution 12 - Amazon Cloudformation

I went to the stack resources tab and checked why some of them couldn't be deleted then I deleted them manually first.

Attributions

All content for this solution is sourced from the original question on Stackoverflow.

The content on this page is licensed under the Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) license.

Content TypeOriginal AuthorOriginal Content on Stackoverflow
QuestionJustinView Question on Stackoverflow
Solution 1 - Amazon CloudformationSimon BuchanView Answer on Stackoverflow
Solution 2 - Amazon CloudformationmysView Answer on Stackoverflow
Solution 3 - Amazon Cloudformationf01View Answer on Stackoverflow
Solution 4 - Amazon Cloudformationuser2981810View Answer on Stackoverflow
Solution 5 - Amazon CloudformationEric NordView Answer on Stackoverflow
Solution 6 - Amazon CloudformationtalentedmrjonesView Answer on Stackoverflow
Solution 7 - Amazon CloudformationJourneyToJsDudeView Answer on Stackoverflow
Solution 8 - Amazon CloudformationasdfqwerView Answer on Stackoverflow
Solution 9 - Amazon CloudformationharryhareView Answer on Stackoverflow
Solution 10 - Amazon CloudformationVijeth KashyapView Answer on Stackoverflow
Solution 11 - Amazon CloudformationwjordanView Answer on Stackoverflow
Solution 12 - Amazon CloudformationAmrView Answer on Stackoverflow