UBIFS failure on many modules [ANSWERED]

Moderators: Oren.R, Eran.M

peter.hall@rvl.co.uk
Posts: 4
Joined: Wed Jun 29, 2016 12:19 pm

UBIFS failure on many modules [ANSWERED]

Postby peter.hall@rvl.co.uk » Thu Jul 27, 2017 11:27 am

We are having issues with corruption within UBIFS (see kernel log below). I cannot recreate the condition here, but our customer has now seen 4 modules with this issue and they are only testing a total of 20 so it is becoming a real problem. All the modules can be recovered by re lashing, but this is not acceptable in the field. In all cases the unit appeared to be operating correctly and when the power was cycled the kernel stops during boot with the UBIFS error. We are looking into a mini UPS, but I am not sure if the power cut is really causing the issue as I can’t recreate it, over many attempts.

I know this is not a lot of information to go on, but I am enquiring to see if there are any known issues in this build. I have done some research and the only thing I can find is the ‘unstable bits issue’ but I don’t think we should see this so regularly.

Any help you could provide would be greatly appreciated

Module: VAR-SOM-DUAL_1000C_1024R_512N_4G_AC_EC_TP_WB_CT_LO_REV1.02
Yocto: Fido


UBIFS: recovery needed
UBI warning: ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 2828:4096, read only 126976 bytes, retry
UBI warning: ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 2828:4096, read only 126976 bytes, retry
UBI warning: ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 2828:4096, read only 126976 bytes, retry
UBI error: ubi_io_read: error -74 (ECC error) while reading 126976 bytes from PEB 2828:4096, read 126976 bytes
CPU: 1 PID: 1 Comm: swapper/0 Not tainted 3.14.38-6QP+g2977c7d #1
[<80015688>] (unwind_backtrace) from [<80011758>] (show_stack+0x10/0x14)
[<80011758>] (show_stack) from [<8073daf4>] (dump_stack+0x7c/0xbc)
[<8073daf4>] (dump_stack) from [<803f2160>] (ubi_io_read+0x128/0x304)
[<803f2160>] (ubi_io_read) from [<803efcac>] (ubi_eba_read_leb+0x134/0x3cc)
[<803efcac>] (ubi_eba_read_leb) from [<803eee14>] (ubi_leb_read+0xa8/0x130)
[<803eee14>] (ubi_leb_read) from [<80208208>] (ubifs_leb_read+0x28/0x8c)
[<80208208>] (ubifs_leb_read) from [<8020fb10>] (ubifs_start_scan+0x74/0xec)
[<8020fb10>] (ubifs_start_scan) from [<8020fdf8>] (ubifs_scan+0x28/0x344)
[<8020fdf8>] (ubifs_scan) from [<802106c4>] (ubifs_replay_journal+0x324/0x1370)
[<802106c4>] (ubifs_replay_journal) from [<8020642c>] (ubifs_mount+0xee0/0x15dc)
[<8020642c>] (ubifs_mount) from [<800dd6ec>] (mount_fs+0x14/0xcc)
[<800dd6ec>] (mount_fs) from [<800f4d2c>] (vfs_kern_mount+0x4c/0xec)
[<800f4d2c>] (vfs_kern_mount) from [<800f7518>] (do_mount+0x1bc/0x9fc)
[<800f7518>] (do_mount) from [<800f80ac>] (SyS_mount+0x84/0xb8)
[<800f80ac>] (SyS_mount) from [<80a2df94>] (mount_block_root+0x10c/0x230)
[<80a2df94>] (mount_block_root) from [<80a2e35c>] (prepare_namespace+0xa8/0x1d4)
[<80a2e35c>] (prepare_namespace) from [<80a2dce4>] (kernel_init_freeable+0x1c8/0x1d8)
[<80a2dce4>] (kernel_init_freeable) from [<807390a0>] (kernel_init+0x8/0xe8)
[<807390a0>] (kernel_init) from [<8000e4f8>] (ret_from_fork+0x14/0x3c)
UBIFS error (pid 1): ubifs_scan: corrupt empty space at LEB 2647:126285
UBIFS error (pid 1): ubifs_scanned_corruption: corruption at LEB 2647:126285
UBIFS error (pid 1): ubifs_scanned_corruption: first 691 bytes from LEB 2647:126285
UBIFS error (pid 1): ubifs_scan: LEB 2647 scanning failed
VFS: Cannot open root device "ubi0:rootfs" or unknown-block(0,0): error -117
Please append a correct "root=" boot option; here are the available partitions:
1f00 2048 mtdblock0 (driver?)
1f01 2048 mtdblock1 (driver?)
1f02 6144 mtdblock2 (driver?)
1f03 514048 mtdblock3 (driver?)
1f04 16384 mtdblock4 (driver?)
1f05 28672 mtdblock5 (driver?)
1f06 475136 mtdblock6 (driver?)
b300 7634944 mmcblk1 driver: mmcblk
b318 512 mmcblk1rpmb (driver?)
b310 4096 mmcblk1boot1 (driver?)
b308 4096 mmcblk1boot0 (driver?)
Kernel panic - not syncing: VFS: Unable to mount root fs on unknown-block(0,0)
CPU0: stopping
CPU: 0 PID: 0 Comm: swapper/0 Not tainted 3.14.38-6QP+g2977c7d #1
[<80015688>] (unwind_backtrace) from [<80011758>] (show_stack+0x10/0x14)
[<80011758>] (show_stack) from [<8073daf4>] (dump_stack+0x7c/0xbc)
[<8073daf4>] (dump_stack) from [<80014610>] (handle_IPI+0x144/0x158)
[<80014610>] (handle_IPI) from [<80008598>] (gic_handle_irq+0x58/0x5c)
[<80008598>] (gic_handle_irq) from [<80012240>] (__irq_svc+0x40/0x70)
Exception stack(0x80a8ff18 to 0x80a8ff60)
ff00: 80a8ff60 3b9aca00
ff20: 233a0955 00000001 80a9d914 80a9d8c8 ab7350d0 00000001 1f6f7b18 00000001
ff40: 80a8e000 00000000 00000017 80a8ff60 00000009 805178f8 00000113 ffffffff
[<80012240>] (__irq_svc) from [<805178f8>] (cpuidle_enter_state+0x54/0xe4)
[<805178f8>] (cpuidle_enter_state) from [<80517a84>] (cpuidle_idle_call+0xfc/0x14c)
[<80517a84>] (cpuidle_idle_call) from [<8000f07c>] (arch_cpu_idle+0x8/0x44)
[<8000f07c>] (arch_cpu_idle) from [<8006bbfc>] (cpu_startup_entry+0x100/0x14c)
[<8006bbfc>] (cpu_startup_entry) from [<80a2db10>] (start_kernel+0x350/0x35c)
[<80a2db10>] (start_kernel) from [<10008074>] (0x10008074)

Eran.M
Posts: 140
Joined: Tue Jan 26, 2016 10:49 am

Re: UBIFS failure on many modules

Postby Eran.M » Thu Jul 27, 2017 12:03 pm

Hi Peter.
Which of our kernel commits are you based on?
I recommend using the latest one of the branch you are using. This is the branch for Yocto Fido:
https://github.com/varigit/linux-2.6-im ... p_ga_var01

We updated the GPMI NAND driver back in March - it may resolve your issue (assuming you did not take this update already).
Also, as you know, it is recommended to avoid sudden power downs when working with read-write partitions.

Regards,
Eran


Return to “Yocto”

Who is online

Users browsing this forum: No registered users and 3 guests