Swearing by acts of worship is haraam but there is no expiation for it

Question

Praise be to Allah.

It is not permissible to swear an oath except by one of the
beautiful names of Allah or one of His sublime attributes, because the
Prophet (blessings and peace of Allah be upon him) said: “Whoever wants to
swear (an oath), let him swear by Allah or else remain silent.”

Narrated by al-Bukhaari, 2679

The Prophet (blessings and peace of Allah be upon him)
regarded swearing by anyone or anything other than Allah as minor shirk,
because it involves venerating something or someone other than Allah,
because an oath can only be sworn by something that is venerated. The
Prophet (blessings and peace of Allah be upon him) said: “Whoever swears by
something other than Allah has committed an act of shirk.”

Narrated by Abu Dawood (3251); classed as saheeh by
al-Albaani in Sunan Abi Dawood.

Swearing an oath by fasting, prayer or any other act of
worship is swearing by something other than Allah, so it is haraam.

It says in Tabyeen al-Haqaa’iq (3/109):

Swearing by an act of worship is not an oath because it is
swearing by something other than Allah, may He be exalted. End quote.

Ibn al-Humaam said in Fath al-Qadeer (5/71):

Swearing by acts of worship is swearing by something other
then Allah or His attributes, so it is not an oath. End quote.

The one who has done that has to repent to Allah, regret what
he has done and resolve never to go back to it again.

An oath sworn by something other than Allah is not a binding
oath, so it does not have to be fulfilled and no expiation has to be offered
for it.

Ibn Hazm said in al-Muhalla (9/125):

The one who swears by something other than Allah has not
sworn and it is not an oath; it is invalid and nothing has to be done with
regard to it except asking Allah for forgiveness and repenting only. End
quote.

Shaykh al-Islam Ibn Taymiyah (may Allah have mercy on) said:

Swearing by created things such as the Ka‘bah, the kngels,
shaykhs, kings, parents, parents’ graves and so on is an oath that is not
binding and no expiation has to be offered for it, according to scholarly
consensus. Rather it is forbidden according to scholarly consensus, and this
prohibition means that it is haraam according to the more sound of the two
opinions. End quote.

Al-Fataawa al-Kubra, 3/222

Based on this, their is nothing wrong with you doing what you
swore not to do if it was permissible, and you do not have to offer any
expiation, but you have to repent from this haraam oath.

And Allah knows best.