How do I compile the da-ghc fork independent of the SDK build system? (docker container or VM available?)

App Development5 posts356 views1 likesLast activity Oct 2022

mkawaOP

Sep 2022

Hi!

I’m beginning work on a formal verification tool for DAML that is likely to require some additions to the daml surface syntax. However, I am having trouble building the DA fork of GHC due to Haskell toolchain and library versioning issues. Is there a docker container or similar development environment available that allows simpler builds of da-ghc?

In particular, I followed the directions here:

github.com

digital-asset/daml/blob/5c66d53aa8968420a259d20bd10052858b79cedd/ghc-lib/new-working-on-ghc-lib.md

Copyright (c) 2022 Digital Asset (Switzerland) GmbH and/or its affiliates. All Rights Reserved.
SPDX-License-Identifier: (Apache-2.0 OR BSD-3-Clause)

# Working on `ghc-lib`

If you need to build, test, deploy or develop [`ghc-lib`](https://github.com/digital-asset/ghc-lib) as used by Daml and utilizing the Digital Asset [GHC fork](https://github.com/digital-asset/ghc) these notes are for you.

Here are instructions for when working on Daml surface syntax, as implemented in the the digital-assert fork of `ghc`. (linked in via `ghc-lib`, see [ghc-lib](/bazel_tools/ghc-lib/).

### Cloning the digital-assert fork of `ghc`

1. Make initial clone from the main ghc gitlab repo:
```
git clone --recurse-submodules https://gitlab.haskell.org/ghc/ghc.git
cd ghc
```

2. Add the DA fork as a remote
```
git remote add da-fork git@github.com:digital-asset/ghc.git

This file has been truncated. show original

Running the command:

hadrian/build.sh --configure --flavour=quickest -j

within the ghc repository yields a number of cabal errors having to do with toolchain and library versions.

Resolving dependencies...
Error: cabal: Could not resolve dependencies:
[__0] next goal: Cabal (user goal)
[__0] rejecting: Cabal-3.8.1.0/installed-3.8.1.0, Cabal-3.6.2.0,
Cabal-3.6.1.0, Cabal-3.6.0.0, Cabal-3.4.1.0, Cabal-3.4.0.0, Cabal-3.2.1.0,
Cabal-3.2.0.0, Cabal-3.0.2.0, Cabal-3.0.1.0 (constraint from user target
requires ==3.0.0.0)
[__0] trying: Cabal-3.0.0.0
[__1] next goal: base (dependency of Cabal)
[__1] rejecting: base-4.17.0.0/installed-4.17.0.0, base-4.16.0.0,
base-4.15.1.0, base-4.15.0.0, base-4.14.3.0, base-4.14.2.0, base-4.14.1.0,
base-4.14.0.0, base-4.13.0.0 (constraint from project config
/Users/ming/research/ghc/hadrian/cabal.project.freeze requires ==4.12.0.0)
[__1] rejecting: base-4.12.0.0 (constraint from non-upgradeable package
requires installed instance)
[__1] rejecting: base-4.11.1.0, base-4.11.0.0, base-4.10.1.0, base-4.10.0.0,
base-4.9.1.0, base-4.9.0.0, base-4.8.2.0, base-4.8.1.0, base-4.8.0.0,
base-4.7.0.2, base-4.7.0.1, base-4.7.0.0, base-4.6.0.1, base-4.6.0.0,
base-4.5.1.0, base-4.5.0.0, base-4.4.1.0, base-4.4.0.0, base-4.3.1.0,
base-4.3.0.0, base-4.2.0.2, base-4.2.0.1, base-4.2.0.0, base-4.1.0.0,
base-4.0.0.0, base-3.0.3.2, base-3.0.3.1 (constraint from project config
/Users/ming/research/ghc/hadrian/cabal.project.freeze requires ==4.12.0.0)
[__1] fail (backjumping, conflict set: Cabal, base)
After searching the rest of the dependency tree exhaustively, these were the
goals I've had most trouble fulfilling: base, Cabal

note that I’ve installed cabal-3.0.0.0 and ghc 8.6.5/base-4.12.0 via ghcup and have tried entering dev-env prior to running build.sh without much luck. Notably, the instructions in step 6 to --override-repository in bazel to the local checkout does seem to result in a successful build, but I can’t find the resulting ghc binary in order to test ghc --ddump-parse …

I am building on aarch64-darwin, but also have x64 and aarch64-linux available.

Thanks!

cocreature

Sep 2022

What does ghc --version show? It looks like the wrong GHC version is picked up. The GHC fork has a nix config which will give you the right version if you enter it via nix-shell.

mkawa

Sep 2022

aah, so i had ghcup append to path instead of prepend, so it was picking up some part brew-installed ghc 9.2.4 instead of the ghc 8.6.5 x64-darwin binary i forced it to install.

nix doesn’t seem to want to run ghc 8.6.5 on aarch64, either on linux or darwin (haskell.org has no official aarch64 binaries for 8.6.5… the x64 binary runs fine on darwin though via rosetta.)

it looks so far like i’ve hacked up my ghcup environment enough to get the fork building on aarch64-darwin (without nix), but the best longer-term plan is probably just to build on x64 with nix-shell. will update if i hit any other snags. thanks!

mkawa

Sep 2022

that didn’t work. The below is what happens when you try and mix x86_64 and aarch64 binaries in the build environment (nix-shell with UNSUPPORTED… on gives the same error). solution in next post (requires some system-wide configuration of nix).

Error message after building first bootstrapping stage:


<no location info>: error:
    Warning: Couldn't figure out LLVM version!
             Make sure you have installed LLVM 7

<no location info>: error:
    Warning: Couldn't figure out LLVM version!
             Make sure you have installed LLVM 7
ghc: panic! (the 'impossible' happened)
  (GHC version 8.8.1 for aarch64-apple-darwin):
	Failed to lookup the datalayout for arm-apple-darwin; available targets: ["i386-unknown-windows","i686-unknown-windows","x86_64-unknown-windows","arm-unknown-linux-gnueabihf","armv6-unknown-linux-gnueabihf","armv6l-unknown-linux-gnueabihf","armv7-unknown-linux-gnueabihf","armv7a-unknown-linux-gnueabi","armv7a-unknown-linux-gnueabihf","armv7l-unknown-linux-gnueabi","armv7l-unknown-linux-gnueabihf","aarch64-unknown-linux-gnu","aarch64-unknown-linux","i386-unknown-linux-gnu","i386-unknown-linux","x86_64-unknown-linux-gnu","x86_64-unknown-linux","x86_64-unknown-linux-android","armv7-unknown-linux-androideabi","aarch64-unknown-linux-android","armv7a-unknown-linux-androideabi","powerpc64le-unknown-linux","i386-apple-darwin","x86_64-apple-darwin","armv7-apple-ios","aarch64-apple-ios","i386-apple-ios","x86_64-apple-ios","amd64-portbld-freebsd","x86_64-unknown-freebsd","aarch64-unknown-freebsd","armv6-unknown-freebsd-gnueabihf","armv7-unknown-freebsd-gnueabihf","arm-unknown-nto-qnx-eabi"]
CallStack (from HasCallStack):
  error, called at compiler/llvmGen/LlvmCodeGen.hs:96:24 in ghc:LlvmCodeGen

Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug

Error when running Shake build system:
  at action, called at src/Rules.hs:35:19 in main:Rules
  at need, called at src/Rules.hs:52:5 in main:Rules
* Depends on: _build/stage1/lib/package.conf.d/rts-1.0.conf
  at need, called at src/Rules/Register.hs:85:5 in main:Rules.Register
* Depends on: _build/stage1/rts/build/libHSrts-1.0_thr_debug.a
  at need, called at src/Rules/Library.hs:118:5 in main:Rules.Library
* Depends on: _build/stage1/rts/build/cmm/Updates.thr_debug_o
  at cmd, called at src/Builder.hs:272:23 in main:Builder
* Raised the exception:
Development.Shake.cmd, system command failed
Command line: _build/stage0/bin/ghc -Wall -hisuf thr_debug_hi -osuf thr_debug_o -hcsuf thr_debug_hc -static -optc-DTHREADED_RTS -optc-DDEBUG -hide-all-packages -no-user-package-db '-package-db _build/stage1/lib/package.conf.d' '-this-unit-id rts-1.0' -i -i_build/stage1/rts/build -i_build/stage1/rts/build/autogen -irts/. -Iincludes -I_build/generated -I_build/stage1/rts/build -I_build/stage1/rts/build/build -I_build/stage1/rts/build/../includes -I_build/stage1/rts/build/includes -I_build/stage1/rts/build/includes/dist-derivedconstants/header -Irts/build -Irts/../includes -Irts/includes -Irts/includes/dist-derivedconstants/header -I_build/generated -optc-I_build/generated -optP-include -optP_build/stage1/rts/build/autogen/cabal_macros.h -ghcversion-file=_build/generated/ghcversion.h -outputdir _build/stage1/rts/build -Wnoncanonical-monad-instances -optc-Wno-unknown-pragmas -c rts/Updates.cmm -o _build/stage1/rts/build/cmm/Updates.thr_debug_o -O2 -H32m -this-unit-id rts -XHaskell98 -ghcversion-file=/Users/ming/research/ghc/_build/generated/ghcversion.h -Irts -Wno-deprecated-flags -Wcpp-undef
Exit code: 1
Stderr:
<no location info>: error:
    Warning: Couldn't figure out LLVM version!
             Make sure you have installed LLVM 7

<no location info>: error:
    Warning: Couldn't figure out LLVM version!
             Make sure you have installed LLVM 7
ghc: panic! (the 'impossible' happened)
  (GHC version 8.8.1 for aarch64-apple-darwin):
	Failed to lookup the datalayout for arm-apple-darwin; available targets: ["i386-unknown-windows","i686-unknown-windows","x86_64-unknown-windows","arm-unknown-linux-gnueabihf","armv6-unknown-linux-gnueabihf","armv6l-unknown-linux-gnueabihf","armv7-unknown-linux-gnueabihf","armv7a-unknown-linux-gnueabi","armv7a-unknown-linux-gnueabihf","armv7l-unknown-linux-gnueabi","armv7l-unknown-linux-gnueabihf","aarch64-unknown-linux-gnu","aarch64-unknown-linux","i386-unknown-linux-gnu","i386-unknown-linux","x86_64-unknown-linux-gnu","x86_64-unknown-linux","x86_64-unknown-linux-android","armv7-unknown-linux-androideabi","aarch64-unknown-linux-android","armv7a-unknown-linux-androideabi","powerpc64le-unknown-linux","i386-apple-darwin","x86_64-apple-darwin","armv7-apple-ios","aarch64-apple-ios","i386-apple-ios","x86_64-apple-ios","amd64-portbld-freebsd","x86_64-unknown-freebsd","aarch64-unknown-freebsd","armv6-unknown-freebsd-gnueabihf","armv7-unknown-freebsd-gnueabihf","arm-unknown-nto-qnx-eabi"]
CallStack (from HasCallStack):
  error, called at compiler/llvmGen/LlvmCodeGen.hs:96:24 in ghc:LlvmCodeGen

Please report this as a GHC bug:  https://www.haskell.org/ghc/reportabug

mkawa

Oct 2022

The following instructions have worked successfully for me.

install multi-user nix and Rosetta 2 if not already installed

follow steps 1 and 2 from this page:

Evan Relf

Building x86-64 Packages With Nix on Apple Silicon

Recent revisions of Nixpkgs provide good support for Apple silicon (aarch64-darwin), however not 100% of packages work natively as of writing. These simple i...

clone digital-asset/ghc using these instructions:

github.com

digital-asset/daml/blob/6df0c24d787ed5ac1fb0626c0d6a823b6548cde9/ghc-lib/new-working-on-ghc-lib.md

Copyright (c) 2022 Digital Asset (Switzerland) GmbH and/or its affiliates. All Rights Reserved.
SPDX-License-Identifier: (Apache-2.0 OR BSD-3-Clause)

# Working on `ghc-lib`

If you need to build, test, deploy or develop [`ghc-lib`](https://github.com/digital-asset/ghc-lib) as used by Daml and utilizing the Digital Asset [GHC fork](https://github.com/digital-asset/ghc) these notes are for you.

Here are instructions for when working on Daml surface syntax, as implemented in the the digital-assert fork of `ghc`. (linked in via `ghc-lib`, see [ghc-lib](/bazel_tools/ghc-lib/).

### Cloning the digital-assert fork of `ghc`

1. Make initial clone from the main ghc gitlab repo:
```
git clone --recurse-submodules https://gitlab.haskell.org/ghc/ghc.git
cd ghc
```

2. Add the DA fork as a remote
```
git remote add da-fork git@github.com:digital-asset/ghc.git

This file has been truncated. show original

change ghc/shell.nix line 6 to:

pkgs = import nixpkgs { system = "x86_64-darwin" ; }

cat “x86_64-darwin” > ./nix/system.nix

nix-shell

← Back to Discussions