Broadcom BCM63xx

This page provides an overview of the Broadcom BCM63xx series soc , which share many similarities with the BCM33xx SoCs (with the exception of the BCM3302, which is a standalone CPU). The difference between the two being that the DSL core in the BCM63xx is replaced with a DOCSIS/EuroDOCSIS core in the BCM33xx series.

The Broadcom BCM63xx SoC integrates ADSL/ADSL2+ capabilities, routing functions, and support for external Wireless NICs.

This SoC family is widely adopted in xDSL platforms globally and is considered one of the most successful xDSL platforms. Its success is attributed to the ease of transitioning older platforms (e.g., BCM6345) to newer ones with minimal software changes.

Architecture information:

Older BCM63xx SoCs are based on the MIPS32 Big Endian instruction set, with architectural similarities to the R4000 microprocessor.

Newer BCM63xx SoCs have transitioned to the ARMv7a Little Endian instruction set, exemplified by chips like the BCM63138.

• The OpenWrt support for the Broadcom BCM63xx SoC family currently only works with following models:
  • 6318
  • 6328
  • 6338
  • 6345
  • 6348
  • 6358 / 6359
  • 6361 / 6362
  • 6368 / 6369
  • 63167 / 63168 / 63169 / 63268 / 63269
• There are working drivers for USB Host (OHCI and EHCI) and Ethernet under the GPL. USB Device drivers are also supported but only for BCM6368 and newer SoCs.
• → brcm63xx.imagetag

The support for Broadcom 63xx is at this state :

• Runtime detection of the SoC: Full Linux support on which the kernel is running.
• Ethernet / switch: GPL driver.
• USB OHCI, EHCI: GPL driver.
• Watchdog: GPL driver.
• SPI: GPL driver, with minor bugs.
• Dual core: supported in BMC6368/6362/63268 and no support in BCM6358.
• NAND flash chips are supported since r13271 (kernel 5.4).
• SPU (Secure Processing Unit): The Cipher Engine has drivers since Linux kernel 4.11, but still not integrated into OpenWrt/LEDE
• Wifi core: not supported, initial work: WIP: bcm63xx: internal wireless support
• FAP (Broadcom Forwarding Assist Processor) not supported. This looks like some kind of hardware NAT.
• No available drivers (neither binary, nor GPL) for DSL, ATM, VoIP, on-board SLIC/SLAC,

• xDSL and ATM are NOT SUPPORTED. Not by some binary nor are there GPL drivers available!
  • Netgear has released some sources for DSL-driver: DG834GBv4 GPL and closed code
  • https://forum.openwrt.org/viewtopic.php?id=24271
  • https://web.archive.org/web/20160622160227/http://comments.gmane.org/gmane.comp.embedded.openwrt.devel/17440
  • http://www.neufbox4.org/forum/viewtopic.php?pid=15930#p15930
  • https://github.com/cubieb/hg556a_source/tree/master/bcmdrivers/broadcom/char/adsl/bcm96358 here seems to be a quite complete stack for bcm6348/58 kernel 2.6
• → BCM63xx ADSL Support on Linux kernel 2.6.8.1 effort to make it work again on Linux kernel 2.6.8.1 from 2004-08-14

Certain Broadcom SoCs, including the BCM6358, BCM6361, BCM6362, and BCM6368, feature dual cores. However, the BCM6358 is limited to using only one core. While the kernel includes SMP support (see smp-bmips.c), fully utilizing both cores is complex. The second CPU must be explicitly initialized, and the current interrupt (IRQ) code only enables interrupts on the first CPU. Consequently, only userspace processes can access the second core, while all interrupt handlers remain tied to the first core.

→ SMP/CMT Broadcom 63xx

• Download: DG834GBv4 GPL and closed code and help writing specification for the DSL core, the place to host specifications is BCM63xx at Sipsolutions.net.
• Improve the bcm63xx SPI driver.
• Dual core SMP/CMT still needs further work, specially for BCM6358 with a shared TLB. If you know how to get rid of the problem of having a shared TLB between 2 cores, with working code, please contact with developers
  see → TLB exception handlers
• BCM63168 with Kernel 3.4 source can be found here: 100AAJX8_4.16L.02A GPL and closed code.
• BCM63168D0 with Kernel 2.6.30 source can be found here: 100AAPP7D0_4.12L.06B_consumer_release GPL and open source code.

• The third digit, when set to 3 (like in BCM6335, BCM6338) denotes a single-chip and cost-reduction oriented design.
• There are also some other variants like bcm6341, which is a DSP used in VoIP products in conjunction with a BCM6348 SoC.
• The bcm63138 supports G.fast, G.inp and SRA.

VIPT = Virtually indexed, physically tagged

See → http://www.linux-mips.org/wiki/Caches

• SSB: 6348, 6358, 6368
• BCMA: 6318, 6328, 6362, 63168, 63268

Hardware random number generator

Only available in BCM6362, BCM6368, BCM6816. GPL supported. bcm63xx-rng.c dev-rng.c

To take advantage of this hardware feature, rng-tools should be installed.

BCM63xx SoCs have cryptographic hardware accelerators. The Cipher engine accelerates the IPSec protocol by using dedicated hardware blocks. BCM63XX SoCs (all family? ) are implemented with the Encapsulating Security Payload (ESP) and Authentication Header (AH) IPSec protocols:

• AES and DES/3DES hardware encryption and decryption.
  • AES in both Cipher Block Chaining (CBC) mode and Counter (CTR) mode. Can be performed in 128-, 192-, and 256-bit modes.
  • DES, 3DES in Cipher Block Chaining (CBC) mode
• HMAC-SHA1 and HMAC-MD5 authentication in hardware.

This what Broadcom calls SPU (Secure Processing Unit). The driver is available with GPL

http://code.google.com/p/gfiber-gflt100/source/browse/bcmdrivers/opensource/char/spudd/impl2/

The SPU drivers has been added since Linux kernel v4.11 → https://github.com/torvalds/linux/commit/9d12ba8

But there isn't still support for SPU under OpenWrt/LEDE.

Serial Peripheral Interface

Two types of SPI controllers are present in BCM63xx:

• SPI : Not available in 6318, 6328, 6345
• HSSPI: High speed SPI, only available in 6318, 6328, 6362, 63268 SoCs

By default only one or two (more in newer SoCs) Slave Selects are available. Additional Slave Selects are at GPIO lines, but they need to be enabled.

Snippet code example for enabling these extra slave-selects at GPIOs:

/* BCM6348 */
	u32 val;
	/* Enable Extra SPI CS */
	/* GPIO 29 is SS1, GPIO 30 is SS2, GPIO 31 is SS2 */
	val = bcm_gpio_readl(GPIO_MODE_REG);
	val |= GPIO_MODE_6348_G1_SPI_MASTER;
	bcm_gpio_writel(val, GPIO_MODE_REG);
 
/* BCM6358 */
	u32 val;
	/* Enable Overlay for SPI SS Pins */
	val = bcm_gpio_readl(GPIO_MODE_REG);
	val |= GPIO_MODE_6358_EXTRA_SPI_SS;
	bcm_gpio_writel(val, GPIO_MODE_REG);
	/* Enable SPI Slave Select as Output Pins */
        /* GPIO 32 is SS2, GPIO 33 is SS3 */
	val = bcm_gpio_readl(GPIO_CTL_HI_REG);
	val |= 0x0003;
	bcm_gpio_writel(val, GPIO_CTL_HI_REG);
 
/* BCM6368 */
	u32 val;
	/* Enable Extra SPI CS */
	val = bcm_gpio_readl(GPIO_MODE_REG);
	val |= (GPIO_MODE_6368_SPI_SSN2 | GPIO_MODE_6368_SPI_SSN3 | GPIO_MODE_6368_SPI_SSN4 | GPIO_MODE_6368_SPI_SSN5);
	bcm_gpio_writel(val, GPIO_MODE_REG);
	/* Enable SPI Slave Select as Output Pins */            
        /* GPIO 28 is SS2, GPIO 29 is SS3, GPIO 30 is SS4, GPIO 31 is SS5*/   
	val = bcm_gpio_readl(GPIO_CTL_LO_REG);
	val |= (GPIO_MODE_6368_SPI_SSN2 | GPIO_MODE_6368_SPI_SSN3 | GPIO_MODE_6368_SPI_SSN4 | GPIO_MODE_6368_SPI_SSN5);
	bcm_gpio_writel(val, GPIO_CTL_LO_REG);
 
/* BCM6328 */
#define SEL_SPI2                 8
#define PINMUX_SEL_SPI2_MASK     (3 << SEL_SPI2)
#define PINMUX_SEL_SPI2          (2 << SEL_SPI2)
	u32 val;
	/* configure pinmux to SPI extra Slave Select */
	val = bcm_gpio_readl(GPIO_PINMUX_OTHR_REG);
	val &= ~PINMUX_SEL_SPI2_MASK;
	bcm_gpio_writel(val, GPIO_PINMUX_OTHR_REG);
 
	val = bcm_gpio_readl(GPIO_PINMUX_OTHR_REG);
	val |= PINMUX_SEL_SPI2;
	bcm_gpio_writel(val, GPIO_PINMUX_OTHR_REG);
 
/* BCM63268 */
#define GPIO_MODE_63268_HSSPI_SSN4		(1 << 16)
#define GPIO_MODE_63268_HSSPI_SSN5		(1 << 17)
#define GPIO_MODE_63268_HSSPI_SSN6		(1 << 8)
#define GPIO_MODE_63268_HSSPI_SSN7		(1 << 9)
	u32 val;
	/* GPIO 16 is SS4, GPIO 17 is SS5, GPIO 8 is SS6, GPIO 9 is SS7*/   
	val = bcm_gpio_readl(GPIO_MODE_REG);
	val |= (GPIO_MODE_63268_HSSPI_SSN4 | GPIO_MODE_63268_HSSPI_SSN5 | GPIO_MODE_63268_HSSPI_SSN6 | GPIO_MODE_63268_HSSPI_SSN7);
	bcm_gpio_writel(val, GPIO_MODE_REG);

We can locate slave selects on the board by toggling the state of them.

1. Build a firmware with devmem enabled in busybox and kernel
2. Use this script to blink a slave select

   #!/bin/sh
    
   # Toggle the SPI_SS_POLARITY, "blink" the SPI chip select
   # Example: "blink" the chip select 2
   # ./sstoggle.sh 2
    
   #6318
   #SPIBASE=0x10003000
    
   #6328 6362 63268
   SPIBASE=0x10001000
    
   DEFAULT=`devmem $SPIBASE`
   OFF=`printf "0x%x" "$(( $DEFAULT | (1 << $1) ))"`
   ON=`printf "0x%x" "$(( $DEFAULT & ~(1 << $1) ))"`
    
   while true; do
   	devmem $SPIBASE 32 $OFF
   	echo "[OFF]: $SPIBASE 32 $OFF"
   	sleep 1
    
   	devmem $SPIBASE 32 $ON
   	echo "[ON ]: $SPIBASE 32 $ON"
   	sleep 1
   done

3. Use a voltimeter or a led (with a 270 ohm series resistor) to see if the candidate for the SPI slave select on the board blinks

The script is only valid for HSSPI.

General Purpose Input/Output

On bcm63xx boards the GPIOs are used for diferent purposes:

• software leds: the GPIOs are controled by the linux kernel, and can be user configured by using led triggers drivers.
• hardware leds: the GPIOs are multiplexed to act as pure leds controled by hardware. The GPIO functionality is lost, avoiding to control them with OpenWrt. They can monitor LAN activity, serial activity, and so on. They can be software controled again by writing some particular registers of the SoC.
• buttons: configured as inputs, software controled using the polling method. Can be configured by the user to trigger events.
• other hardware: some GPIOs are wired to hardware specific interfaces, such as PCI, PCMCIA, ethernet, UART, SPI, and so on. They are multiplexed and enabled by OpenWrt during initialization of the board devices.

See BCM6348 GPIO pinmux

The amount of GPIOs of each SoC model is different:

When having more than 32 GPIOs they are splitted between 2 gpiochips. The labels in the Linux kernel are:

• bcm63xx-gpio.0
• bcm63xx-gpio.1

A few GPIOs are shared with external IRQs on most SoCs except BCM6338

*) Guessed

Caveats:

• IRQ_EXT_4 and IRQ_EXT_5 aren't defined in the kernel driver
• IRQ_EXT_4 and IRQ_EXT_5 aren't implemented in BCM6358 SoC (OpenWrt ≤ Barrier Breaker). Proposed patch for Barrier Breaker → http://pastebin.com/xaqJznWw
• IRQ_EXT_4 in BCM6348 cannot be managed because it seems there isn't enough CP0 CAUSE registers to do the job.
• In Chaos Calmer version the external IRQs are broken → https://dev.openwrt.org/ticket/21613

Snippet kernel code example: a button press triggers an IRQ, printing something on the console. Tested on BCM6348, Openwrt 12.09 and GPIO33 connected to an external button.

#include <linux/kernel.h>
#include <linux/err.h>
#include <linux/module.h>
#include <linux/spinlock.h>
#include <linux/interrupt.h>
 
#include <bcm63xx_cpu.h>
#include <bcm63xx_io.h>
#include <bcm63xx_regs.h>
#include <bcm63xx_irq.h>
 
static irqreturn_t gpio_interrupt(int irq, void *dev_id)
{
	printk("my IRQ triggered!!!!\n");
	return IRQ_HANDLED;
}
 
int bcm63xx_button_init(void)
{
	int ret, irq;
 
	printk("TEST IRQ (GPIO-button)\n");
	irq = IRQ_EXT_1;
	ret = request_irq(irq, gpio_interrupt, 0, "bcm63xx_extIRQ", NULL);
	if (ret) {
		printk(KERN_ERR "bcm63xx-extIRQ: failed to register irq %d\n",irq);
		return ret;
	}
	printk("Mapped IRQ %d\n", irq );
 
	return 0;
}
 
arch_initcall(bcm63xx_button_init);

bootloader: Some devices use redboot such as Inventel Liveboxes. Most of the others use cfe with a built-in LZMA decompressor. CFE is not using standard LZMA compression arguments, and most noticeably, changes the dictionary size, so beware. Thomson routers have their own bootloader.

There is released source code for RedBoot (Inventel Livebox), and probably can be modified to work with other routers. Also there is some source code for uboot.

On several CPE (Customer-premises equipment) hardware devices and especially on smart phones, the OEM bootloaders are feature poor (no netboot, no booting from a USB stick, etc.), obfuscated (require some magic values to be correct) or completely messed up and make it cumbersome, difficult or impossible to install free software on the device. It is thus paramount to always have at least some products available, that have OEM bootloaders that keep installing free software easy (cf. generic.flashing). And it could be interesting to port such bootloaders to devices, which happen to come with a restricted bootloader. Compare the available bootloader out there, their license, available code and feature sets. Please also remember that available source code it NOT enough, it has to be under some license, that allow for modification and redistribution.

bcm63xx boards have U-boot support thanks to the developer Álvaro Fernández (Noltari). Currently only available for the RAM bootloader version. The ROM version requires low level initialisations to be integrated into U-Boot (TODO).

https://github.com/Noltari/u-boot/commits/master?author=noltari

There is an official broadcom u-boot port for 63137/63138, 63158, and 63178; it is able to replace cfe, but there is no GPL release of this yet.

There exists an utility to backup the entire flash:
cfetool
You must connect your PC with the bcm63xx router via serial TTL port while CFE is running. Then execute cfetool with a command like this, maybe different with different boot address / flash sizes.

./cfetool.py --read=dump.bin --addr=0xB8000000 --size=0x1000000 --block=0x10000
--addr=0xB8000000 -> Flash Memory Address (see CFE bootlog --> Boot Address)
--size=0x1000000 -> 16Mb Flash
--block=0x10000 -> Memory dumped each iteration (default is 10Kb 0x2800)

cfetool expects the serial port used is /dev/ttyUSB0 in your PC, but you can change it with “--serial=/dev/ttyUSB1”.

Note: not all CFEs have internally the dm/sm command, as a result of this cfetool may not work with some devices. Alternatively you can dump the flash via traditional methods like JTAG or with an OpenWrt ramdisk firmware version.

On the BCM6348 the MPI interface is wired to both the flash chip and the miniPCI interface. The CPU clock configuration is strapped from 5 pins on this interface. These 5 pins use pulldown resistors (4.7 or 10 kohm) to configure the CPU clock:

Example of CPU clock modification → Comtrend CT5361 overclocking

Note: only tested on BCM6348KPBG

The same pins used in BCM6348 are also used in the BCM6368 SoC. 4.7 kohm pull down resistors are also used to configure the CPU frequency.

The list of related devices: bcm63xx, bcm6318, bcm6328, bcm6348, bcm6358, bcm6361, bcm6362, bcm6368, bcm6816, bcm6818 bcm63167, bcm63168, bcm63169, bcm63268,