Standard density-based topology optimization results in topologies with a staircased boundary description that often also contains intermediate densities. For problems that are sensitive to the way interfaces are resolved, a smoother description of the boundary during optimization is desired. A level set-based topology optimization approach, combined with the Interface-enriched Generalized Finite Element Method (IGFEM) provides such a smooth boundary description throughout the optimization procedure. Radial Basis Functions (RBFs), used to construct the level set function, offer control of the complexity of the design, similar to filters in density-based topology optimization. Analytical sensitivities for IGFEM are also derived and the method is tested on well-known compliance minimization problems.