If it’s difficult to make the hole bigger you may be able to use a 150 to 100 reducer to fit the 150 hose to your existing vent. Ideally it would be best to ask the air con manufacturer but they may give you a no because it’s not ideal even though it would work with a slight reduction in efficiency.
You could try using a reducer, leave the hose at 150 right up to the vent, and see if it works. If not then you would have no option but to make the hole bigger or run the hose through a window.

My portable air-conditioned packed up last year, and the replacement unit I bought needs a 150mm vent, not the 100mm that the original had.  I tried using a reducer to connect it to the existing pipe and it would barely cool the room at all.  Chucking the pipe out the window, it cooled perfectly well.  I'm gonna suggest that you probably need to enlarge the hole and fit the correct size pipe specified or you will probably be disappointed with the performance of it