Compiling Trouble Shooting: Segmentation Fault and GCC Illegal Instruction
Tags: clanguage, cprogramming, graphdatabase, nebulagraph
Published: 2020-02-12 07:57:57

---
title: Compiling Trouble Shooting: Segmentation Fault and GCC Illegal Instruction
published: true
date: 2020-02-05 04:17:27 UTC
tags: c-language,c-programming,graph-database,nebula-graph
canonical_url: https://nebula-graph.io/en/posts/segmentation-fault-gcc-illegal-instruction-trouble-shooting/
---

![](https://cdn-images-1.medium.com/max/1024/1*M1U5u_2GfAd-jbpvgiqRPw.png)<figcaption>Compiling Trouble Shooting — Segmentation Fault and Illegal Instruction</figcaption>

Recently I have been re-organizing and re-compiling all third-party dependencies of [Nebula Graph](https://github.com/vesoft-inc/nebula), an open-source distributed graph database. And I have come across two interesting issues and would like to share with you.

#### Flex Segmentation Fault — — Segmentation fault (core dumped)

Segmentation fault happened upon compiling Flex：

{% gist https://gist.github.com/Jamie-VESoft/b26fca70b0aa55aaf2027ab812c0de6e %}

Check coredump with gdb:

{% gist https://gist.github.com/Jamie-VESoft/bba727d5532e1fedfffc84de17f7a5b3 %}

We can see from the assembly code above that the issue was caused by the allocate\_array function. reallocarray returned a pointer, which should be saved in the 64-bit register rax. However, allocate\_arraycalled reallocarray and returned the 32-bit register eax. Meanwhile it used instruction cltq to extend eaxto rax.

The possible reason could be that the prototype of reallocarray that allocate\_array saw was different than the real prototype.

When looking at the compiing log, I did find such a warning, like _implicit declaration of function_ _reallocarray'_.

This issue can be resolved by adding CFLAGS=-D\_GNU\_SOURCE at the configure stage.

Please note that this issue is not supposed to appear every time. However, enabling compiling/link option -pie and core parameter kernel.randomize\_va\_space helps produce the issue.

**Takeaways:**

1. The return type of an implicit declarative function is int in C
2. Pay attention to compiler warnings with -Wall and -Wextra enabled. Better enable -Werror under development mode

#### GCC Illegal Instruction — — internal compiler error: Illegal instruction

A while ago I’ve received feedback from Nebula Graph users that they encountered a compiler error: ileggal instruction. See the details in this pull request: [https://github.com/vesoft-inc/nebula/issues/978](https://github.com/vesoft-inc/nebula/issues/978).

Below is the error message:

{% gist https://gist.github.com/Jamie-VESoft/912a0d6a2de5c5995bb2b3b49a243649 %}

Since it’s an **_internal compiler error,_** my assumption would be that an illegal instruction was encountered in g++ itself. To locate the specific illegal instruction set and the component it belongs to, we need to reproduce the error.

Luckily, the code snippet below can do the magic:

{% gist https://gist.github.com/Jamie-VESoft/12a5538831069f75f9734390dec89237 %}

Illegal instruction is sure to trigger SIGIL. Since g++ acts only as the entrance of the compiler, the real compiler is cc1plus.

We can use gdb to perform the compiling process and catch the illegal instruction on spot:

{% gist https://gist.github.com/Jamie-VESoft/9c3c7cd514ab9bfe9152b69b75b6be78 %}

Bingo！

mulx belongs to BMI2 instruction set and the CPU of the machine in error doesn't support this instruction set.

After a thorough investigation, I found that it was GMP, which is one of GCC’s dependencies, that introduced this instruction set. By default, GMP would detect the CPU type of the host machine at the configure stage to make use of the most recent instruction sets, which improves performance while sacrificing the portability of the binary.

To solve the issue, you can try to override two files in the GMP source tree, i.e. _config.guess_ and _config.sub_ with \_configfsf.guess\_ and _configfsf.sub_ respectively before **configure .**

### Conclusion

- GCC won’t adopt new instruction set due to compatibility issue by default.
- To balance compatibility and performance, you need to do some extra work. For example, select and bind a specific instance for gllibc when it is running.

Finally, if you are interested in compiling the source code of Nebula Graph, please refer to the documentation [here](https://github.com/vesoft-inc/nebula/blob/master/docs/manual-EN/3.build-develop-and-administration/1.build/1.build-source-code.md).


* * *