The number of Fox colorings, as well as branched coverings and quantum invariants, were used as obstructions to tangle embeddings (see, for example, [Kre99,KSW00,PSW04*,Rub00]). Quandle cocycle invariants can be used as obstructions as well [AERSS].
For a tangle , the cocycle invariant is defined by the formula similar to the case of knots, where, for the purpose of applications to tangle embeddings, we require that the colors on the boundary points of are monochromatic (have the same color). Such a coloring is called boundary monochromatic.
Suppose a tangle diagram embeds in a knot (or link) diagram . For any boundary monochromatic coloring of , there is a unique coloring of such that the restriction on is the given coloring of ( ) and that all the other arcs (the arcs of outside of ) receive the same color as that of the boundary color of . Then the contribution of to agrees with the contribution of to , since the contribution from the outside of the tangle is trivial, being monochromatic. Hence we obtain the condition , where the multi-subset is defined as follows. If an element is repeated times in a multiset, call the multiplicity of , then for multisets , means that if , then and the multiplicity of in is less than or equal to the multiplicity of in .
In [AERSS], tables of tangles [KSS03] and knots [Liv] were examined. For those in the tables, the cocycle invariants were computed, and compared to obtain information on which tangles do not embed in which knots in the tables. For one case it was possible to completely determine, using the cocycle invariants, in which knots up to crossings a tangle embeds.