Is it meaningful to identify interaction effect when a main effect in a model (main effects only, without interaction terms) is not significant?

Let's say there's one DV (Y) and three IVs (X1, X2, X3), and among IVs, X1 is a dummy variable.In the regression model without interaction terms, the results can be represented like this:

Y ~ X1 + X2 + X3X1 : non-significantX2 : significantX3 : significant

In this case, is it meaningful to check some interaction terms (e.g. X1 $\cdot$ X2 or X1 $\cdot$ X3)? At first I thought I don't have to because the main effect of X1 indicates non-significance. But I'm afraid I'm missing something important.