As to the first question: There are several reasons for this. One is rather inherent and can be understood by considering the following analogy:

Java, C, and many other programming languages are like chess: There are many syntactic rules, and by learning them, you already obtain a rough overview of what you can do in principle. You try out these constructs, and get a sense that you have accomplished something, even if it is rather worthless, and more complex tasks are extremely hard to carry out successfully in these languages.

Prolog is more like Go: The syntax is very simple, and there is essentially only a single language element, the logical rule. This means that even if you know, syntactically and semantically, almost everything about the language, you have no idea what to do at first. This can be rather frustrating. From this, beginners easily arrive at the misguided conclusion that the language is useless, or restricted to very specific applications. But it only means they have not grasped its true power and flexibility! Getting to the core of Prolog is hard, and requires systematic guidance.

This inherent difficulty is frequently compounded by a rather ineffective and outdated didactic approach which, at its worst, stresses difficult and mostly superseded procedural aspects over more important declarative principles and more modern solutions like constraints. This easily gives the misguided impression that the language is rather imperative and limited in nature, and again causes many students to dismiss it due to their wrong impressions.

A third reason is found in the implementational complexity: From a user's perspective, a major attraction of Prolog is its ease of use due to the syntactic simplicity, powerful implicit search mechanism, generality of predicates etc. which are features that are rather specific to logic programming languages. The complexity of all this is shifted to the implementation level: In order to make all this both powerful and efficient, the implementation must do many things for you. This means you need, among other things and in no particular order: an efficient garbage collector, JIT indexing, a fitting virtual machine architecture, a fast implementation of unbounded integers, rational numbers, good exception handling, ISO compliance, many goodies like tabling, an efficient implementation of constraints over integers, Boolean variables, Herbrand terms etc. Most of these topics are even now still subject of active research in the logic programming community, with different advantages and trade-offs. Implementing an efficient Prolog system is a project that easily takes 30 to 40 years. In fact, we are only now getting to the point where systems become sufficiently robust and feature-rich to run complex client/server applications for months and years. In such complexities, you find the answer why Prolog isn't more popular yet. It has simply taken a few decades to implement all this in satisfactory ways, and this work is still ongoing. In my view, Prolog is now becoming interesting.

To the second point, Prolog already is a great general-purpose language. You can use it for almost all applications that are currently written in Java and Python, for example. Of course, there are always some features that are worth adding on top or via extensions, and certain tasks would benefit from this. For example, you can add extensions for type checking, and for fast arrays. Various Prolog implementations are already experimenting with such extensions. Many extensions can in fact be implemented via term and goal expansion, a facility that is analogous to macros in Lisp, or via simple reasoning over given programs.